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THE COMPLEXITY OF FINITE FUNCTIONS 


ABSTRACT 


Lower bounds on the length of formulas for finite functions are 
obtained from a generalization of a theorem of Specker. Let f: 
{0,1,...,d-1}" 4 {0,1,...,d-1} be a function which can be represented 
by a formula of length S cen. For any m, if n is sufficiently large, 
there is a restriction f': {0,1,...,d-1}™ 4 (0,1,...,d=1} of £ which 
is representable by a special class of formulas called homogeneous 
e-complexes. By showing that certain functions do not have restric- 
tions representable by homogeneous e-complexes, we are able to conclude 
that the length of formulas representing the mod p sum, p 3d, or the 
connectedness of a pattern on a discrete retina cannot be bounded by 
a linear function of the number of variables in the formula. 


Also considered are perceptrons over finite fields (cyclic per- 
ceptrons). It is shown that cyclic perceptrons of bounded order 
cannot represent the geometric predicate connectivity. An interesting 
aspect of this is that one proof of the corresponding result for 
bounded order perceptrons over the rationals rests on the inability 
of the latter to represent the parity function. However, the parity 
function requires order 1 if the field has chracteristic 2; thus, 
this proof breaks down in the case of cyclic perceptrons. Another 
geometric predicate that cannot be represented by bounded order 
cyclic perceptrons is Euler number equals k (for an arbitrary k). 
However, this predicate can be represented by bounded order percep- 
trons over the rationals. It must be noted, however, that our proofs 
are different and much simpler than the corresponding proofs derived 
by Minsky and Papert for perceptrons over the rationals. 


Finally, ye investigate k-pattern spectra of a discrete retina. 
This is the 2k “tuple, each component of which corresponds to the 
number of times a particular kxk pattern occurs on the retina. it 
is shown that the only topological predicates that can be determined 
from k-pattern spectra of discrete figures are functions of the Euler 
number of the figure. 


This report reproduces a thesis of the same title submitted to 
the Department of Electrical Engineering, Massachusetts Institute 
of Technology, in partial fulfillment of the requirements for 

the degree of Doctor of Philosophy, February 1972. 
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CHAPTER ONE 
INTRODUCTION AND SURVEY 


1.1 Finite Functions 

Let n be nonzero and finite; then a partial function IN™ + IN , defined 
on only finitely many n-tuples, is called a finite function. We will restrict 
our attention to a subclass of finite functions. D = {0,1,...,d-1} is an 
initial interval of JN. Then we will consider #, the set of all (total) 
functions D" 4D for all possible D and (finite and nonzero) n. 

Let f: p* +D. Then f is identified with a (functional) table with a* 
rows (corresponding to all possible n=tuples over D) and n+l columns (corres- 
ponding to the n arguments and the value of f). Obviously the number of func- 
tions D" 4D is ae, 

Consider any function f: D” 4D for arbitrary D, n. We will say that f 
depends on the qth argument if and only if there exist two n-«tuples 
a= (ay sees 2s vee era) and b = (Dy sees sds sees sbi) such that a, = b_ for j #i, 


J 


a; # by» and f(a) # f(b). Suppose that £ does not depend on its j®" argument; 


then we will say that the ic argument is a fictitious argument. 


1.2 Formulas 

Let there be given the countable sets = = (Xp »Xyoeee) of variable symbols 
and Q of operator symbols. Each element of 9 is a name for a function in F, 
and conversely each function in * has a name in © Let 9 € ] represent the 


function f: D’ 4D. Then we will write arg(?) =n and dom(?) =D. 


1.2.1 Definition 
A D-formula is a finite expression F = P(G,s+2+5G) such that 0 € 2, 


arg(o) =n, dom(~) = D, and either G € = or G, is a D-formula for 1 <i <n, 


A formula is simply a D-formula for some D. 

Let F be an arbitrary D-formula and let x be the highest numbered 
variable symbol appearing in F. Then F represents a function f: D+ D. 

This correspondence is well-known and we will not describe it in detail. 
Without danger of imprecision, F will also be considered as a representation 
for all functions obtained from f by adding fictitious arguments. 

Let there be given two formulas F and G. Suppose that F represents a 
certain function f, and also a representation for f can be obtained from G 
by possibly choosing different variable symbols. Then we will say that F 
is equivalent to G (F = G). 

Remarks. Usually, if we are dealing with D-formulas for a single domain 
D, we represent the identity function by a variable symbols (i.e., we omit 
the operator symbol for the identity). In the formal model we use, we cannot 
do this since it would be ambiguous. Also, for purely technical reasons, we 
insist that every operator has at least one argument (otherwise, the wording 
of several definitions and results would be more cumbersome). Thus, we do 
not allow constants. Rather, instead of constants, we use operators with ote 
fictitious argument. Suppose we are given the formula F. Occasionally, we 
will say “Replace the variable x (in F) by the constant a". This is to be 
interpreted as "Replace the variable x with a(y) " where y is a variable 


symbol not appearing in F, 


Let f: D" 4D and let g be an arbitrary finite function of n arguments 
with domain E © D" and such that g=f E. Let F be a D-formula for (i.e., 
representing) f. Then we can also say that (F,E) represents g. From now on 
we will not be pedantic, and we will simply say that F represents g. Some of 
the main results in this thesis are concerned with the question, given a 
specific function D' 4D » how much can we simplify its representation if we 
choose an E~formula for it with D F E. 

If F is an arbitrary formula, then the set of variables appearing in it 
will be called its support (denoted by S(F)). The set of operators appearing 
in F will be called its basis (denoted by B(F)). 

Let 3¢ ©. ‘Then the set of formulas F such that B(F) © $ will be called 
the set of formulas over ¢. Hopefully without too much danger of ambiguity, 
we will also say that $ is a basis of operators (for formulas over $). All 
the significant results we will describe deal with formulas over ¢ when $ 
is finite (and representing a set of operators with domain D for a single 
value of D). From now on, whenever a basis of operators $ is introduced, 
it is always assumed finite. Usually, we are interested only in bases that 
allow all function D° 4D for a certain D and arbitrary n to be represented. 
Such bases will be called complete bases (for D). 

Notation. Elements of ¥ will always be denoted by lower case Latin 
letters. The various bases of operators we will use will be denoted by 
capital Greek letters; operators (i.e., basis elements) will be denoted by 
lower case Greek letters (except for well known operators for which established 
notation exists); formulas will be denoted by capital Latin letters; and D 


will always refer to the domain of formulas. d will denote D. 


1.2.2 Example. 


If D = {0,1}, then the functions Dp" 4 D for arbitrary n are known as 
Boolean functions. A complete basis for {0,1} conists of the binary operators 
A (conjunction) and V (disjunction), and the unary operator ™ (complementation). 
This basis shall be denoted by IJ. The formula F = V(AC™ (5) 2X5) AG = OQ) )) 


over Il represents x @ XxX, (the mod 2 sum of xy and x5). Usually, this is 


1 
written as x, Ax, Vx, Ax,. We have S(F) = (X) 2X4], and B(F) = Il, 


1 2 1 2 
A convenient representation of formulas is by trees. This is a standard 
device that will not be described; suffice it to say that to each formula F 
there corresponds a tree T(F) whose terminal nodes are labelled with variable 
symbols and the nonterminal nodes with basis symbols. As an example, let F 
be as defined in Example 1.2.2. Then T(F) is shown in Fig. 1.1. 


Given a formula F, we need a notation for subformulas of F. 


The definition of subformula is the standard one: (1) F is a subformula 


of F, (2) if F = OCF, se+e oF) then if F, for 1 <i <k is not a variable 


symbol, any subformula of F, is a subformula of F, and (3) subformulas of F 


i 
are only objects satisfying (1) and (2). Subformulas distinct from F are 
proper subformulas. 

Let G be a subformula of F such that G = KH, »+++ Hy). Then we will say 
H, =G for 1 <i S %. This notation can be iterated. In Example 1.2.2, 
F 5.2 = ~(%5)- However, note that Foo. is a variable symbol which according 
to our definition is not a formula. This can be remedied by replacing this 
particular occurrence of the variable symbol xy by id(x,). For this reason 


we will require that all the bases we consider contain the identity function 


whether this is specifically mentioned or not. 


If G= Fy(1).§(2)e00§ (0)? then j = j(1)j(2)...j(r) is called the index 
of G (for completeness, let \ denote the index of F). If G is a proper sub- 
formula of F, then F = H(X,G) where X U S(G) = S(F) and H(X,z) is a formula 
(determined by j) where z appears only once. We write H = F/G. In this case, 
with F and G as given, we will also write S%(G) =X (i.e., the variables of 
F that appear outside of G). We define S#(F) = ¢. The subscript F will 
generally be suppressed when it will be clear to what formula F we refer to. 
In what follows, whenever we will deal with a subformula G of F, it will be 
assumed that the index of G is also given; for if not, then, e.g., F/G and 
S*(G) are not uniquely defined. 

Frequently, formulas will occur where certain variables have been re- 
placed with constants. Suppose F is a formula over 3, X © S(F), and a € D; 
then, F with all variables except those in X replaced by a will be denoted 
by Be Obvicusly, F. is a formula over $ U{a}. If f is an arbitrary function, 
X a subset of its arguments, then ae has the analogous meaning, viz., the 
function obtained from f by restricting the elements outside of X to a. 

The functional table of is obtained from that of f by deleting all columns 
except those that correspond to X and retaining only the rows with a 


entries in the deleted columns. 


1.3. Measures of Complexity 


Let us introduce the three most widely studied measures on formulas: 


(1) Length. The length of a formula F, denoted by L(F), is the number of 
occurrences of variable symbols in F. In other words, it is the number of 


terminal nodes of T(F). 


10 


(2) Cost. The cost of a formula F, denoted by C(F), is the number of opera- 


tor symbols in F. In other words, it is the number of nonterminal nodes of 


T(F). 


(3) Depth. The depth of a formula F, denoted by D(F), is the depth of nest- 


ing of operators in F. In other words, it is the number of arcs on the long- 


est branch of T(F). 


A . P n Saude . 
Now, given an arbitrary function £: D 7% D and a (finite) basis #, we 


define the length of f over $ as 


L(£,$) = min({%: There exists a formula F over & for f such that 
L(F) = £}) 


If f cannot be represented by a formula over $, we define L(f£,%) =% Simil- 


arly, for cost and depth. 


It is noteworthy that all the measures above are closely related. In fact, 


Co LCE, §) < Cc (F,6) <¢,-L(£,§) (1.3.1) 


cy‘ log, (L(E, )) < D(f£,4) <s c,* log, (L(£,%)) (1.3.2) 


for an arbitrary function f such that L(f,4), C(£,5), and D(f,%) are finite, 
and certain constants Cos Cys Cos and Cy that depends on §. The basis 3 is 
also arbitrary, except in the case of the right inequality of (1.3.2) where 
it must be such that all the constants and the function g (see Lamma 1.3.1) 


may be represented. 


We first establish the relation between cost and length (1.3.1). 


11 


Any formula F over % can be built up from one which uses only one opera- 
tor symbol (an elementary formula) by successively replacing variable symbols 
with new elementary formulas. If F does not contain one-argument operators, 
then whenever we increase the cost during the build-up (by adding an elementary 
formula with cost 1), we also increase the length. Specifically, the length 
increases by between n_, -1 and n -l where n ,. andn are respectively 

min max min max 
the smallest number larger than 1 and the greatest number of arguments of an 


operator of $6. This results in the estimate 


L c 
WaT *L@) $ C@) s Ty VLE) Clesad) 
max min 


where c = 1, Suppose F contains one-argument operators. In other words, T(F) 
contains nodes with branching factor one. Let the maximal number of such nodes 
that occur one after another on any branch of T(F) be c*; then (1.3.3) still 
applies with c =c*¥ +1. (1.3.1) is obtained from (1.3.3) by noting that the 
minimal length or cost representation of any function (over the chosen basis 
$) can be achieved with a formula where c* s ae (the number of functions DD). 
The left inequality in (1.3.2) is established by a trivial counting argu- 
ment (the maximal number of terminal nodes in a tree with branching factor < 
ore and depth d is a: The right side requires more effort (the following 


argument is due to R. W. Floyd). We first state the following obvious 


1.3.1 Lemma. 
Given a formula F such that F = F, (X) F,(%,)) where F, is a proper sub- 


formula of F and Fy = F/F,, the following holds: 


12 


F= F,(F, (Co), Fy (Xj, )o0005F, XC 35)F, Xp) 
where Cc, for D Si Sd-1 is any formula representing the constants 0,...,d-1 
(or, as we have remarked previously, the one-argument function with constant 
value), and F3 is any formula representing the function B(ZqseeesZa 4 oZy_ = 


w = . = = ~ 
Z if z4 = 0; z, if 245 Lyeewey ae] if za = d-1. 


1 


Let F be an arbitrary formula over 3, and let G be a proper subformula 
of F. We already know that F = H(X,G). The claim is made that if L(F) s 1, 


G can be chosen in such a way that 


n 
max 

L(H)-1,L(G) € FAT *L(F) (1.3.4) 
max 


where ees is as defined previously. (Remark: L(H)-1 is the number of 
occurrences of the variables of S*(G) in H.) 

To find G use the following procedure: Start with F and proceed to sub- 
formulas of F. Assume you are considering the subformula K. Then two cases 
can arise, Either among K for 1 = j =k where k is the number of arguments 
of the outermost operator of K there is one, j', such that LK 51) 2 a L(F) 
(0 <a@<¢1 will be determined later with the purpose of obtaining the lowest 
possible estimate of L(H)-1 and L(G)), or not. In the first case, proceed to 
Ky and containue. Otherwise, set G = Kye where j" is such that LK jn) = 


max (L(K >? and terminate. Before the procedure terminates, L(K) 2 a@-L(F). 
1<j<k : 


Thus 


< L(G) < O° L(F). This also means (1-@)*-L(F) S L(H)-1 < (In 


max max 
~L(F) (because L(G) + L(H)-1 = L(F)). The lowest bound for L(G) and L(H) 


) 


is obtained by setting @ = l- ~ ; hence (1.3.4) 
max 
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L P. is of 


depth c, depending on 6. If the outlined procedure is applied recursively to 


Now apply Lemma 1.3.1 with G replacing F, and H replacing F 


2 


H(X,C,) for 0 si Sd-1 and to G, we obtain in (1.3.2) Cy eee where 


‘ log, 


Note that unlike the cost-length relationship, the minimal value of 
depth may not be achieved by the same formula as the minimal value for length. 

Apart from the relationship between the various measures, depth and cos” 
will not be treated further. Even though in what follows (in this chapter) 
many things hold mutatis mutandis for depth and cost, most of the specific 


discussion and the examples shall be confined to length. 


b d_to ngth su 
In this section we will mention several questions that have been asked 
about the complexity of finite functions, their status as of this writing, 


and how they relate to the work to be described here. 


A more precise expression is obtained if the right side of (1.3.2) is re- 
placed by tee slog, (L(£))+4 for some constant £. Namely, if we start out 
with a formula F and decompose it according to (1.3.4) and Lemma 1.3.1, then 
the length of H(X, C,) and G is bounded by SF) +k where k is the length of C,- 


After n applications of Lemma 1.3.1, the lengths of the relevant formulas are 


bounded by on L(F) + kT +.o.00+ Inte ~ oe (L(F)) + + ; hence the figure 
1- = 
b 


above. 
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The Problem of Aggregate Length 

Let $ be a complete basis (for a certain domain D). The statement of the 
problem is: What is the largest number L(n,%) such that there exists a function 
f: D4 D and L(£,%) = L(n,5)? 

It has been studied by several authors, and is now effectively disposed 
of. Riordan and Shannon [Ri42] first derived a lower bound for L(n,J). 
Actually they studied series-parallel contact networks, but the two models 
are equivalent. The first upper bound (for the same model) was obtained by 
Shannon [Sh49]. Krichevskii [Kr59] derived a lower bound for L(n,%) for 
arbitrary domains and bases, while Lupanov [Lu59] obtained the best upper 
bound for the general case. The result is 


qm 
log n 


O(L(n,8)) = (1.4.1) 


where O(f(n)) = g(n) means that lim a is finite and nonzero. There 
nwo 


are two remarks that are in order here. The first is that 
Formulas represent finite functions efficiently; i.e., 
the total number of formulas (over a given basis $) of 
length up to L(n,$) closely matches the number of func- 
tions of n variables. (1.4.2) 


The second is 


F : n 
The fraction of functions D ~D that can be represented 


by formulas of length up to L(n,$)-(1-¢€)for an arbitrary 


0 < ©< 1 approaches zero as n 7% (1.4.3) 


The interested reader may obtain more information in the literature cited 


above. 
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Obviously, we could define functions C(n,$) and D(n,%) analogous to 
L(n,$) in terms of the cost and depth measures. In general, such functions 
(aggregate complexity functions) can be defined in connection with any model 
for the representation of functions a) and any measure on this model (an 
obvious variation of L(n,$) is to remove the condition of completeness on $4), 
It should be noted that the asymtotic behavior of aggregate complexity functions 


remains an active area of research. For references on the subject, see Lupanov 


(Lu70]. 


The Minimization Problem 

Investigation of the complexity of finite functions started on representa- 
tions of Boolean functions by logical circuits. In fact, formulas can be 
thought of as circuits with fan-out one. Thus, the first problems studied 
were those a logic designer is likely to ask: Given a finite function, what 
is the minimal circuit (formula) that represents it (i.e., find the complexity, 
and do so "effectively"). 

Unfortunately, no statisfactory solution to the minimization problem 
exists (for any measure). This does not mean that it is impossible to obtain 
a minimal formula for a given function f: Dp’ + D; rather that existing 
algorithms are impractical. Thus, it is always possible to order formulas 
according to length, and then search all formulas up to length L(n,$) for the 
first formula that represents f; but since there are ao functions of n argu- 
ments this approach is absurb. 

At the present, all existing algorithms for the minimization of functional 
representations employ some sort of an exhaustive search (e.g., the Quine 


algoxithm for the minimization of disjunctive form representations of Boolean 
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functions). In fact, there is reason to believe that a more efficient method 


does not exist, i.e., 


1.4.1 Conjecture 
Any generally applicable exact minimization procedure is comparable (in 


terms of computational complexity) to an exhaustive search among formulas. 


It is useful to consider a specific machine model. Let us consider 
implementations of such a procedure as a deterministic one~tape Turing machine 
M, (see, e.g., Arbib [Ar69]) that receives as its input the d"-tuple defining 
an arbitrary function f: Dp’ + D, and whose output is the minimal formula F 
(over $) for f. Conjecture 1.4.1 gives us that the computation time of 
Ms may attain an exponential (in the length of the input). Let us venture 


a more restrictive and precise version of Conjecture 1.4.1: 


1.4.2 Conjecture 


Let My be as described, m is the length of its input, and let 1(m) be a 
function such that Am. +0 asm +® for an arbitrary constant cs 1. Then 
c 


the proportion of inputs of length m at which the running time of M exceeds 


N\(m) approaches 1 as m 7%, 


Actually, the specific machine model on which the procedure of Conjecture 
1.4.1 above is implemented is not particularly important. It can be easily 
shown (see, e.g., Arbib [Ar69], Chapter 4) that different deterministic machine 
models (this applies to the most widely used models, e.g., one-tape and multi- 


tape Turing machines) can simulate each other in such a way that the running 
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time of one is related to the running time of another at most by a polynomial. 
In this way, whenever the running time is exponential (in the length of the 
input) in one case, it must be so also in others. 

It seems that Conjecture 1.4.1 was first expressed by Yablonskii [Ya59]. 
A very interesting result connected with this subject was recently obtained 
by Cook [Co71]. He obtained strong evidence that a simpler problem requires 
nonpolynomial time. The problem is that of recognizing whether a certain 
disjunctive normal form (for a Boolean function) represents the constant 1. 
Cook showed that if this problem could be solved in polynomial time (by a 
deterministic one-tape Turing machine), then a number of other problems that 


are regarded as very difficult (e.g., given the graphs G, and G,, determine 


1 2? 


whether Gy is isomorphic to a subgraph of G,; the recognition of primes; etc.), 


93 
would also be rapidly computable. Note that a fast minimization machine 
would give us also a fast constant recognizer; hence, Cook's results supports 


Conjecture 1.4.1. 


The Classification Problem 


In view of the difficulty of finding an exact nontrivial solution to the 
minimization problem (i.e., one that does not employ exhaustive search), present 
research is directed at establishing bounds for the length of functions. We 
consider sequences of functions Eyoeee of 1,... arguments and study the 


growth rate of the length of f Thus, we can talk of classes of linear 


45 
(length) sequences, quadratic (length) sequences, etc. Also of nonpolynomial 
(length) sequences. Unfortunately, if a sequence belongs to a nonlinear class, 


it is very difficult to estimate its length. We cannot even assign represen~ 


tatives to the polynomial classes of degree > 2, let alone the nonpolynomial 
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classes. In fact, at the present we have only a very limited store of examples 


of nonlinear sequences, 


n 
Consider the Boolean function = = 8 Xie Subbotovskaya [Su61] gave a 


striking proof that O(L(E, M1) 2 ae It was known already to Shannon (see 


[Sh49], or [Ya54]) that ocL(e2,0)) s ne (the length of this sequence, of course, 
grows linearly if © is used). Unfortunately, it seems that the technique of 
fSu61] cannot be generalized to dy 2. Subbotovskaya's result has recently 
been improved by Khrapchenko [Kh71]. He succeeded in showing that O(L(E., 1I)) 

2 n’, Since this result employs a very interesting technique, and since it 


has not yet been translated into English, it is reproduced in Appendix B. 


Neciporuk [Ne66] discovered a sequence of Boolean functions fo such that 
2 

n 
logn 


O(L(E »$)) = for an arbitrary basis 5. It is true that the functions 
involved in the Neciporuk sequence are rather "artificial" in that, while 
defined in a straightforward way, they have no special significance; however, 
lately Harper and Savage [Ha71] have succeeded in applying the Neciporuk tech- 
nique to a practical combinatorial problem (The Marriage Problem). 

Neciporuk's construction is based on the following lemma: Let f be a 
Boolean function of n arguments. Consider a subset X of the arguments of f 
and the set of restrictions of f to X obtained by setting the arguments outside 
of X to constants. Let the number of such restrictions be r. If F is any 
formula over a finite basis % for f, then the number of occurrences of variables 
representing the arguments in X is 2 c+logor where c depends on the basis $ 
(for the proof of this see [Ne66] or [Ha71]). 

The Neciporuk function fo of n arguments is then obtained as follows: 
The n arguments are arranged in a rectangular array with dimensions as shown 


in Fig. 1.2. Each argument x,. is associated with a 0-1 valued m-tuple a., 
ij ee Ea | 
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such that (1) not all components are 0, and (2) if (i,j) # (k,4£) then 
aay # ayy Then we define 

f = © x, ® K(a,.,k) 

Pcie mec ne 


a whose second 


where Ka; 5 >*) denotes the conjunction of those arguments Xy 


subscript (t) corresponds to nonzero components of 33° 
It can be verified that the number of restrictions of f. to the variables 
of an arbitrary row (except, perhaps, the last which may be imcomplete) ob- 
tained by replacing the variables of the other rows with constants is 7 eas 
This follows from the fact that any Boolean function can be uniquely represen- 
ted by a Boolean polynomial (see Lemma 4.5). Then, by the lemma above, the 


number of occurrences of variables of any row (except, perhaps the last) in 


any formula for f. is 2 c+-(n=m); hence, the length of f over xcs BR (n-m). 
n i n m 


n2 
logon 


In other words, O(L(@_ »9)) = for an arbitrary basis %. 
Neciporuk's construction may be viewed as a solution to a special case 


of the following problem (the problem of exhibiting a function of arbitrary 


length): Given a basis $ and a number k < L(n,#), exhibit a function 
n2 
logon 


f: DY +p of length 2 k over $. In Neciporuk's case O(k) = 3 
Since so few examples of functions that are known to be of large length 
exist (in spite of (1.4.3)), the reader has no doubt already gained the 
impression that this problem too is very difficult. However, we again have 
the trivial solution that consists in examining formulas in n variables in 
the order of their length, recording the functions they represent, and 
choosing the first previously unencountered function represented by a formula 


of length 2k. In fact, it is reasonable to state an analog of Conjecture 


1.4.1: 
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1.4.3 Conjecture 
The problem of exhibiting a function of arbitrary length is comparable 


(in terms of computational complexity) to an exhaustive search among formulas. 


We again make this conjecture more precise on the example of determin- 


istic one=tape Turing machines. 


1.3.4 Conjecture 


¢ is an arbitrary basis. Nz is a deterministic one-tape Turing machine 
with input (n,k) where n is arbitrary and k < L(n,%), and whose output is the 
d”-tuple describing a function f of n arguments such that L(f,4) 2k. Then 
there exists a constant c >1 such that if k 2 €*L(n,$) for any 0 < <1, 


n 
the running time of Ns on input (n,k) exceeds c° when n 2 n( ©), 


We can sum up the discussion of the classification problem as follows. 
The problem is far from understood. At the present no sequence of functions 
is known whose length grows faster than A Isolated examples of sequences 
with growth rate < 2 are known, and present research is directed at inven~ 
ting more general techniques that can be used for estimating the complexity 
of whole classes of sequences. Also techniques have to be devised for d > 2. 


The importance of this will be discussed below in Section 1.5. 


1.5. Specker's Theorem 


The first general technique for proving the nonlinearity of a large 
class of sequences (of Boolean functions) was discovered by Specker [Ho68]. 


Let the basis I] U {x ® y} be denoted by =. Then 
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Theorem (Specker). 


If f is a Boolean function of n arguments, if L(f£,2) <c-n for some 


constant c, then for any integer m, if n 2 Tg myc), a subset X = (Xj, .¢0+5%] 


of the arguments of f can be found such that (1) 


x m m 
£5 = coe c,° I (1 @ x.) @ Co = x; (1.5.1) 
i=1 i=1 


where Co» Cys Co are Boolean constants and Tg (m,c) is a certain number- 
t 
theoretic function. Furthermore, (2) if the basis is Il (the other assump- 
tions remaining unchanged), then Cy = 0. 
This theorem has been used by Hodes and Specker to show that the predi- 


cate 


n 
>» x 0 mod k (1.5.2) 


t=1 


for k y 2 and x, ¢€ {0,1} is of nonlinear length over %. 


i 
Using the second statement of the theorem, they are also able to give 

an alternative proof of the nonlinearily of the length of 8 x, Over Il, 
Another result obtained with Specker's Theorem is the fade that some 

geometrical predicates (in particular, connectivity) discussed by Minsky 

and Papert [Mi69] are of nonlinear length over & (see Hodes [Ho70]). 
In Chapter Two we will formulate and prove a generalization of Specker's 


Theorem (Theorem 2.2.2) to include the case d > 2 and multi-argument oper- 


ators in ¢. Our proof reveals the nature of both results more clearly. 


+ 
Te Will be discussed in Chapter Two. 
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They belong to a class of combinatorial results reminiscent of Ramsey's 
Theorem (see Ryser [Ry63]). In fact, an earlier version of our proof 

of Theorem 2.2.2 used Ramsey's Theorem. Besides this, Theorem 2.2.2 
enables us to derive the nonlinearity of new functions (sequences of 
functions) such as counting mod p where p is a prime, d possibly equals 

p, but there are restrictions on the basis, etc. An example of an im- 
provement over existing results is the connectivity predicate. Hodes 
{[Ho70] proves that it is nonlinear if d = 2. However, in Automata Theory, 
for example, the result that a certain language can be computed in non- 
linear time if k states are used in the finite control would be considered 
weak. Rather we search for proofs that work for arbitrary finite controls. 
The Generalized Specker Theorem (Theorem 2.2.2) gives us a tool for proving 
the nonlinearity of the length of the connectivity predicate regardless of 
the domain D and basis $ We can apply it to conpectivity by "reducing" 
connectivity (for the meaning of “reduction” see [Mi69] or 3.2) to certain 
symmetric functions. 

We should note that the generalization of Specker's Theorem that we 
prove is the obvious one to attempt; but, as the reader will see, the proof 
turns out to be less straightforward. As an indication, consider (1.5.2). 
It does not generalize directly to d > 2 since, e.g., the function {0,1} m4 
{0,1} defined by x 


fai. * 
d = 3. This is because 


= 0 mod 6 can be represented in linear length with 


n n n 
[ © x, = 0 mod 6] = [ =x, = 0 mod 3] A[ =x, = 0 mod 2] 
f=1 * i=1 del * 
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Hodes and Specker do not derive any bounds for the lengths of the 
functions investigated by them. This question is asked (and to an extent 


answered) in 3.3. 


1.6 Cyclic Perceptrons 


Cyclic Perceptrons will be treated in Chapter Four. They are an 
application of ideas of Minsky and Papert to the representation of functions 
by combinations of finite operators. In particular, one of the concerns 
in [Mi69] is to formalize the intuitive idea that the connectivity predicate, 
being "global" in nature, cannot be computed (or represented) by a "simple" 
combination of "local" predicates. 

The perceptron is the predicate 

Ae a,"%, 20 
where I is an indexing set, a; € Q, the rationals, P, € $, a set of Boolean 
functions (whose value is interpreted as being either the rational 0 or 1. 
The cyclic perceptron is defined as 

A ay Ds EY 
where a, € F, a finite field, Y S F, and other symbols have the same inter- 
pretation as before. Thus, both represent a certain Boolean function. 

Minsky and Papert introduce the concept of the order of a perceptron 
(the maximal number of arguments on which 0, depends where i ranges over I). 
They define then the order of a predicate as the minimal order of a per- 


ceptron that represents the predicate. They formalize "local" by defining 
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an infinite predicate sequence to be local if and only if every member is 
representable by a perceptron of order <r, for some finite r. They are 
then able to show that connectivity is nonlocal. 

The concept of order can also be applied to cyclic perceptrons. Chapter 
Four will contain results on the order of the various predicates introduced 
in [Mi6é9]. In particular, connectivity is shown to be nonlocal. This will 
be an extension (to finite fields of arbitrary characteristic) of the results 
described in [Vi70]. 

Chapter Five describes a model of computation (Pattern Counting Machines) 
that again performs a "local" computation followed by a "global'' computation. 
In this case the "local" computation is even more constrained than in the 
case of perceptrons. The result is that no matter how cleverly we utilize 
the "local" information in the subsequent "global" phase, the connectivity 


predicated cannot be computed. 
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T(F) for the formula F in Example 1.2.2 


Fig. 1.1 


_th 
j column 


,th 
i” row 


number of columns: m = logan! +1 
number of rows: [n/m 
The array of arguments used in the definition of the 
Neciporuk function fo 


Fig. 1.2 
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CHAPTER TWO 


A GENERALIZATION OF A THEOREM OF SPECKER 


2.1 e-Complexes 

Throughout this section, all formulas are D-formulas for some fixed (but 
arbitrary) domain D, and all operators are functions DY 4D. 

Given the formulas FyoeeeoFL, we shall call the formula F = O(Fys+++5F_) 
where © is an arbitrary operator a parallel combination (PC) of Fy ,+++,F. o is 
called the decoding operator of F,. 

Let F(X,z) be a formula where the distinguished variable z appears only 


once, and let G be an arbitrary formula. Then F(X,G) shall be called a series 


combination (SC) of F and G through z. 


2.1.1 Definition 


We give an inductive definition of an elongated n-component (e component ) 
for n 2 OQ, 
(1) Let Pen be an arbitrary unary operator and zanarbitrary variable symbol. 


Then 9,64) is an e,~component. z is the input variable while Orn is the 


input operator. 


(2) Let © be an arbitrary binary operator, G an arbitrary e-] component , 
and x ¢ S(G). Then F = 9(x,G) (or 9(G,x)) is an e "component. The input vari- 
able and input operator of G are also the input variable and input operator 
of F. x is a lateral variable of F. Any lateral variable of Gis also a 


lateral variable of F. © will be called an internal operator of F. 
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An example of an e,7component is given in Fig. 2.1. Let F be an arbitrary 
e"component , and let x be the sequence of lateral variables arranged in the 
order they are connected to the branch of T(F) extending to the input variable. 
Then x is the lateral sequence of F. If F is an €,)7component , then the lateral 
sequence of F is \ (the empty sequence). For example, the lateral sequence of 
the e=component in Fig. 2.1 is Xpoeee eke 


An e "component with all internal operators equal is a homogeneous 


e component ° 


2.1.2 Definition 
A formula F is an e” “complex if (1) F is a PC of the e "components Frocees 
Fie and (2) the lateral sequence of Fy for 2 <i <r is either equal to the 


lateral sequence of Fie or the reverse of it. 


F F are the components of F. If the variables of F, are numbered 


uf 


as in Fig. 2.1, the second condition of Definition 2.1.2 means that any compo- 


qretee 


nent Fyoeee oF either appears as in Fig. 2.1, or as in Fig. 2.2. The compo- 
nents of the former kind will be known as standard components, while those of 
the latter kind will be called the reverse components. The lateral sequence 
of Fi will also be called the lateral sequence of F. 

Both in the case of e “components and e--complexes, one or both indices 
will occasionally be omitted if the particular property they refer to is 


irrelevant to the argument at hand. 


An e-complex composed of homogeneous e-components is a homogeneous 


e~comp lex. 
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One might wonder what the purpose of introducing e-complexes is since 
for appropriate r and m every function of n variables can be represented by 
an e. ~complex. Thus, it would seem that this class of formulas is trivial. 
However, we will be concerned with er ~complexes where r remains fixed as n 
grows without bounds, and this will allow us to obtain interesting results. 

We introduce some notation. Let F be an component with lateral 

i 


sequence (1) "4 (2)? a! a € D is an arbitrary constant. ° denotes 


the internal operator corresponding to x, 


£4)" Also set o 41 = Pine Then 


(a, 


ere) if 1 Sj <k < ntl 


a che Po i as 
2 = s is Ss nt 
wey 9, tf 155 =k sat 


undefined otherwise 


Note that aa 4) is a unary operator if j = n+l, otherwise it is a binary 
> 
operator (if it is defined at all). Usually we will suppress the superscript 
a because it will be clear what constant is referred to. 


We now state the simple 


2.1.3 Proposition 

Let F be an e "component with lateral sequence x and input variable z. 
y is an arbitrary subsequence of x of length m 2 0 and a € D is an arbitrary 
constant. If we denote the set consisting of z and the elements of y by Y, 


then FY is equivalent to an e "component G. 
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Let x = x x eee XX ) and a x, x eee ok, = Xe Set i(0 = 0 


and i(m+1) = n+l. Then G has the operators 5 = fori<sj< 


Piet 1, £(4)) 
m+1 CW atl is the input operator of G). OQ 


2.1.4 Remark 


Obviously, Proposition 2.1.3 holds for e=complexes as well; one merely 


has to perform the above construction for each component. 


Proposition 2.1.3 will be frequently invoked. Namely, we will take an 
e~complex F, select a subsequence y & x, the lateral sequence of F, and obtain 
G as above. In this case, Gis called the result of an a-merger with basis 
y on F. 


We introduce another restricted class of formulas. 


2.1.5 Definition 

A series parallel combination of e-components (SPCeC) is obtained accor- 
ding to the following rules: 

(1) An e=component is an SPCeC. 

(2) Let F and G be an e-component and an arbitrary SPCeC respectively. 
Then the SC of F and G through the input variable of F is an SPCeC. 

(3) If F peee oF are SPCeC's, then a PC of F 


at 1 
(4) An SPCeC is only an object satisfying (1), (2), or (3). 


geee ok is an SPCeC. 
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Given an arbitrary SPCeC F, we describe its set of components. If F 
consists of the single e=component G, then G is the only component of F. 
If F is the SC of an e-component G and another SPCeC H, then the set of 
components of F consists of G and the set of components of H. If Fis a 
PC of FyoseesFis then the set of components of Fconsists of the sets of 
components of Fy for 1 SisSr. Among the components of F, those whose input 
variable corresponds to a terminal node of T(F) will be called terminal 
components while the others will be called internal components. An example 
of an SPCeC is given in Fig. 2.3. This particular SPCeC has four terminal 


components and two internal components. 


2.1.6 Proposition 
An SPCeC is equivalent to a PC of r e=components where r = d°I+J and I 


and J respectively are the number of internal and the number of terminal component of F. 


Proof F can be converted into a PC of e-components by using Lemma 1.3.1. 
The estimate of the number of e=components in the PC is also obtained from 


there. qn 


Remark It is a simple matter to verify that if F of Proposition 2.1.6 


has k components, then I s k-1; and thus r = d> (k=1)+1. 


2.1.7 Proposition 


F is a SPCeC with k components FyoceeoFye Fy for 1 <i skis an en 
component for n 2 0, and, furthermore, the sets of lateral variables of Fy 
and ue are equal for 1 si, j sk. Let X be the set of lateral variables 


of Fy and let Z be the set of input variables of F. Then for any m 2 O and 
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ae¢eDifin= 1, (ak) where 11, (ak) is a certain function (to be defined), 
YUZ 


there exists a subset Y © X with ly| =m such that Fo is equivalent to an 


xr . 
e complex G with Y as the set of lateral variables. Furthermore, r * d+ (k-1)+1. 


Proof If m = 0, we can immediately apply Proposition 2.1.6 and obtain an 
xr 
€,7 complex where r is as described in the statement of the proposition; thus 


71, (05k) = 0. We assume, therefore, that my 0. 


We recall the following familiar result: 

Let i(1), £(2) peesk C@RID A) be a sequence of distinct integers, Then 
we can extract a subsequence of length p that is either increasing or decreasing 
(for the proof see Berge [Be71] p. 16). 

Without loss of generality, we can assume that the lateral sequence of 
Fy is Kypoere sks Then the lateral sequence of Fy is X51)? By (o) oa 
The sequence i(1), i(2),..., i(m) consists of distinct integers; therefore, 
ifne2 (n,-1) "41, we can apply the above result and find a subset X) SX of 
n, variables such that after performing an a-merger with basis x on all 
components of F, the lateral sequences of the descendants of Fi and Fy are 
either the same or opposite. We can continue in this way, processing one 
after another all components. We end up with an SPCeC with components 
Gy reer G such that the lateral sequence of G; for 2 si <k is either equal 
to that of Gy or the reverse of it. To obtain G, we apply Proposition 2.1.6. 
In order that ly| =m, we must have 


gkn1 
2 7, m,k) = (m1) 


tao} 


form 21. The estimate for r is obtained from the Remark following 


Proposition 2.1.6. O 
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Another equivalence that will be used later is given by 


2.1.8 Lemma 

E is an e~comp lex, X is the set of its lateral variables, and Z is the 
set of its input variables. Then for any m 20 and a€D, if n2 Ty (mr), 
there exists a subset Y © X with !Y] =m such that pw is equivalent to a 


r 
homogeneous e comp Lex F. 


Proof If m = 0, we simply use Proposition 2.1.3, and the result is a homo- 
geneous €q~comp lex (obviously, any €)7 complex is homogeneous). Then 1, (0,4) =0, 
Thus, from now on we assume that m 2 1, 


The proof will be given for the special case when E has two components: 


a standard component Ey and a reverse component E,. It will then only be 


2 


indicated how to generalize the proof. 
A procedure (The Homogenizing Procedure=-HP) will be described that will 


2 pone 
transform an ea omens G consisting of a standard component G) and a reverse 


component Gp with p 2 Na (q) (for a function us that will be defined later) 
and with the properties: (1) There exist (possibly empty) subsets Ry and 


R, © D such that o, (asy) h R, = id (identity on R,) and i yt R, = id 


2 R 


Ry 2 


for 1 Si S p where O; and vy is an operator of G) and Gy respectively 


and the first argument corresponds to the lateral variable, and (2) 
VL sig <6) [o, (,y) ER, > 0, Gy) = 0. (x,y) ] (2e431) 


(i.e., the operators of Gy are identical on the inverse image of Ry). Simil- 


arly for the operators of Gy on Roe 
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Remark Note that if R, and Ro include the range of every operator, 


Property 2 translates into the identity of the operators. In particular, 


this holds if R, =R 


1 =D. 


2 
2 
Remark Note that an arbitrary e =complex satisfies Properties 1 and 2 
with Ry = R, = go 
The result of applying HP will be an e_rcomplex H that will either be 
homogeneous, or will have Properties 1 and 2 with sy and Ss, replacing Rg, and 


= 
Ry respectively and R, aa Sy or R, ¥ 85. Due to the Remarks above and to 
the fact that D is finite, repeated application of HP on E finally yields F. 


The condition on n is 


n2 T, (m, 2) = T1, (1, (. «1, (m)...)) (2. TZ) 
SO -—” 


2d times 


This bound for n corresponds to the worst case when R, or Ry increase by only 


1 
1 on each application of HP. 

Before describing HP, note the useful fact that because of Property l, 
Properties 1 and 2 are preserved under a-mergers. 

Description of HP The lateral sequence of G is of length (vtl)-u~1 for 


certain values of u and v that will be defined later. 


Consider the sequence 


a a 
OC Ccet)euttbeu)? Y ((ve1)eutl, (vektL)+u)? (2.1.3) 


for k = 1,...,v. Sequence (2.1.3) is illustrated in Fig. 2.4. The two 


vertical lines represent G, and G 


1 3 the numbered horizontal outlets represent 


the lateral variables (with the corresponding number); the boxes indicate the 
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variables and operators that take part in the formation of any particular 
O,, ., and t,. .,3; an 'x' beside a variable indicates that it is not set to 
(i,3) (i,9) 

the constant while ‘a' indicates that it is set to a; the two checked boxes 


represent the first member of (2.1.3). 


In the sequence (2.1.3), either (Case I) the ranges of ° ((ke-1) -utl ety) 
? 


and Gai io GD) for 1 <k < v are included in Ry and Ry respectively, 


or (Case II) not. 

Case I. If v is large enough, we can find q identical elements in the 
sequence (2.1.3). Let the indices k corresponding to these elements be 
Kianteek a Performing an a-merger with this set as basis, the desired 
e-complex H is obtained. Note that in this case we use Property 1 of G. 
Namely if o* is the first component of a pair in (2.1.3) whose range & Ry 
and if © is an arbitrary operator of G)> then 9(a,o*) = O* (similarly for 
the second components of the pairs in (2.1.3) and operators of G,)- Thus, 


the components of the identical pairs in (2.1.3) become the operators of H. 


a* 2 ae 2 
A bound for v is q:(d ) (d is the number of operators D' 7D). 


Case II. Assume ® ((he1) y bse) ¢ Ry for some b, c € D and 


sutl, feu 


1<%£v_ (the case when VOye y 5) é R, can be treated 


S)eutl, (v-£+1)+u 
similarly). But then 


Ph uny, fey) Ose) ¢ Rg, (2.1.4) 


for all 0 S$ j S u-l (as a consequence of Property 1). Provided that u is 
large enough, we can find an element e € D that appears w times in the se- 
quence (2.1.4). Let the indices j corresponding to the appearances of e be 
j(1),..+,j(w) (see Fig. 2.5). Obviously then (all the variables considered 


except Xp. have been set to a), 
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Phun 5 (t), feu-4(t-1)-1) 42) =e 


for 2st <w (the first argument of Py 3) corresponds to the lateral 
? 


variable). Thus, 


PC heu-4(t), feueg(te1)-1) ot Ry U {e} = id 
At this point we consider separately two cases: 


Case IIa 4 = 1 and j(w) = u-1l. We perform an a-merger with the basis 
consisting of the variables with indices u-j(t)-1 for 1<t <w-1l. As a re- 


sult of this we obtain an en -complex G' such that Property 1 holds for G) 


1 1 
on Ry U fe} (G, satisfies Property 1 on Ry © R,; in any event R,-R, = ¢). 
Note that at this point Property 2 is still satisfied only on Ry and R, by 


G) and G, respectively. 
Case IIb. £#1 or j(w) e¢ u-l. We perform an a-merger with the basis 


consisting of the variables with indices £-u-j(t)-1 for 1 St Sw. As a result 


rt 


2 
of this we obtain an e, comp Lex G" such that Property (almost) holds for G, 


(the same remarks regarding G,"and Property 1 as well as G", Gl" and Property 
BORD 1° "2 


2 apply as in Case IIa). The only exception may be oF (the operator of G 


: F octet ro 
that is closest to the decoding function). By definition, oF Ce en CS oa 


and there is no assurance that oy (ase) =e. We may rectify this situation 
by absorbing oY into the decoding function inthe following way: Let G™ = 


8c(G", Gh). Now set xy =a (after the a-merger the variables have been 


renumbered). Let S(G") = {x} =U. Then we have (G™) =/G' os S(of(a,G)), one 


9 with the input operator modified 


where Gy equals Gy minus 04 and G, equals G 


as follows: aa = (a, u') emember that Gy is a reverse components, and, 


2 : ‘ 
hence, x, is attached to w. Clearly, G' is ane eee satisfying 


1 a 
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Property 1 with RQ, U {e}] replacing Rie 
We can now resume considering Cases IIa and b together. To obtain H 
(with components Hy and H,) we must find among O; for 1 s i s w-1 q operators 

that are identical on the inverse image of Ry U {e] (i.e., (2.1.1) with 
Ry U {e] replacing R,) and again perform an a-merger. We again emphasize 


that the operators of H, are identical only on the inverse image of Ros and 


2 
this property has not been violated by any of the transformations of the 
original e-complex G. 

To obtain q operators that are identical on the inverse image of R, U fe}, 


2 
it is sufficient that w-l 2 qud® ; therefore, u 2 Ce a +1) and 


3 (a) = qesd+ 6 tqed + (8°46)5d=1 (2.1.5) 


2 
where 6 = at - This is obtained from the values of u and v derived above. 


Recall that T, (@) = (vtl)*u-1, the length of G. uy for r = 2 can then be 
obtained from (2.1.5) and (2.1.2). 
The proof for the general case is obtained by defining the Generalized 


H Procedure (GHP) with the corresponding function 7 We consider instead 


4° 
of (2.1.3) the sequence 
1 x 1 o" 
Crop yet Oe ay? Vest eryeree hes ey? 

' ! al t 

where s = (k-1)*utl, t = keu, s' = (v-k)-utl, and t' = (v-k+t1)-u. Oy poe sO; 
re 

denote the operators of the standard components of G while Vee, denote 
the operators of the reverse components (r' +r" =r). Without detailed 


r ¢ ° 
argument we state that in the general case v = q-§ while u remains the same 


(u is determined by the requirements of Case II at which time only one 
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component is considered). From this we obtain analogously to (2.1.5) 
2 
Ty (a) = qed. 8  tgeas (8 48)4d-1 
TN, (m,r) can then be obtained from 


1, @,r) = To te Gee) 


Ly 


red times 


which is an analog of (2.1.2). 


2.1.9 Remark 

As we have seen, 1, in Lemma 2.1.8 depends on r, the number of components 
of F. However, we shall mostly be using e-complexes that contain many components 
that are identical except for the input operator (such e=complexes are obtained 
e.g., by the use of Proposition 2.1.7). It may be checked that in the application 
of GHP only one representative from each such group of components need be 
considered. This significantly reduces 1: Similarly, in computing No from 
(2.1.5), only £-d compositions are required where £ is the number of groups of 
similar components (corresponding to d compositions for each group of similar 


components). 


2.1.10 Remark 

The operators of a homogeneous e" ~complex F obtained as a result of 
applying Lemma 2.1.8 possess an added property that will be used later: Let 
R be the range of (x,y) an internal operator of R (x corresponds to a lateral 
variable); then o (a,y) R= id, (the identity 0M R). This fact follows from 
the definition of HP (GHP). This particular property of the operators of 


the components of F will be called the t*-property relative to y. In what 
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follows, we will always suppress "relative to y" since there is no danger 
of ambiguity. We will simiarly suppress the subscript R unless we will be 
interested in a specific range. We will abbreviate "F is an e-component 
(complex) whose operators possess the 1?~property" to "F is an e-component 
(complex) with the t*=property". 

A familiar and convenient way of representing a binary operator (x,y) 
is by a labeled directed graph. The graph of , denoted by ['(2), is defined 
as follows: The nodes of I(©) are labeled with elements of D. A directed 
arc labeled with a € D exists from b to c if and only if (a,b) = c. 

If D={1, 2, 3, 4} and R = (2, 3, 4} an example of a graph I() for 
an operator with the Ip-property is shown in Fig. 2.6. 


Given an arbitrary e "component F, the output of the operator O. is 
Oe Me hcg Hogg rts 9, Hy Pin (VI D+ + 0) 


in the case of a homogeneous e., "component with internal operator © this will 


be abbreviated to 


OCR Hye 1X5 0,67). 


2,2 The Generalized Specker's Theorem 


We first give the following 


2.2.1 Definition 
Let % be an arbitrary basis and a € D any constant. Let F be a formula 
over § with S(F) = X UY U Z such that Ix| < Tax (the maximal number of 


arguments of an operator of %), Y is disjoint from X, but otherwise arbitrary, 
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and Z = {z} is a singleton (disjoint from X and Y) such that z occurs only 
once in F. The set of functions f(X,z) represented by all possible such 
formulas F with the elements of Y replaced by the constant operator a will 


be denoted by $7, 


In particular, every operator of $ with all but k 2 1 arguments (k is 
arbitrary) replaced by a is in 5°, Note that if o(X,z) € $7 then © may 
qualify for 37 by virtue of a number of different representations. If z 
(or any variable of X) in any one of them corresponds to a variable that 
occurs only once, it is called a distinguished argument). The other arguments 
are called free arguments. Thus we may easily find a basis $ and an operator 
(9 such that all arguments are at the same time distinguished and free. 

We now define a restricted class of e-components and e-complexes: $ is an 
arbitrary basis and a € D is any constant. Let F be an arbitrary €,)7 component; 
then F is an @g "component over o°, Let w(x,z) € 5° be a binary operator, z 
a distinguished argument (hence x is free), and G an ey-] Component over 5°. 
then ©(x,G) is an e,"component over 5°. An e-complex over 6° is an e-complex 
such that all its components are e=components over 57, 


The main result of this chapter is 


2.2.2 Theorem 

Let there be given the function f: D” 4D such that L(f,%) S c.n for some 
constant c and basis $. Then for anym 21andacDifne2 1], (c sm), there 
exists a subset Y of the arguments of f£ such that ly| =m and e is either a 


r ‘ a. f 
constant or is represented by F, a homogeneous e ~complex over €2 with the I -property, 


Y as the set of lateral variables, constant input operators, and r < d: (2c-1)+1. 
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Proof 
If f has m fictitious arguments, then let Y be the set of these arguments, 
and f is a constant. From now on we assume that f has <= m-1 fictitious 
arguments, 
The statement of the theorem gives us that there exists a formula E 
over $ such that L(E) S cen. Therefore, there are 2 1/2-n variable symbols 
representing the arguments of f which either do not appear in E or appear 
Ss 2*c times. In other words, there are 2 1/2n-m+l variable symbols that 
actually appear in E and such that the number of occurrences of each is S 2°c. 
Denote the set of these variables by X- 
Tf n2 2+7N,(ny,2c m-1, we can apn’ Lemma A.9 and obtain a subset 
2 


G = 
X, x with Ix, | n, and such that E. 


$* with at most 2c components and such that the set of lateral variables of 


is equivalent to Ey» and SPCeC over 


every component is Xo- 


If ny 2 7, (ng »2c) we can apply Proposition 2.1.7 and obtain a subset 
xX 


r 
x and such that E> is equivalent to E,, ane. “complex 


3 3 3 
over 8° with r < d*(2c-1)+1). The estimate for r is obtained at this point. 


<x, with |x,| =n 


If n, 2 Tm, 2c) we can apply Lemma 2.1.8 and Remark 2.1.9 and obtain 
a 
F, the desired homogeneous e_ complex over 3° with the I “property. The 


i property is a consequence of Lemma 2.1.8. 
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Discussion of Ts The present proof yields 
1, (mye) = 2+ (TN, (Ny (m, 2c), 2c), 2c)+m-1 


The exact representation of 1, is extremely complex, and in what follows we 
shall use only a very rough approximation. In Appendix A it is seen that 
Tg (tk) (as a function of k) grows faster than iexp(b,2k) for any constant b. 
The functions Ny and No contribute only insignificantly to this, and thus we 
state: 
Ti, (mc) 2 iexp(b,4c) for ¢ 2 c(b) 
C252 kd 
and an arbitrary constant b 
(Later we shall se that the size of uP prevents us from obtaining any 
interesting bounds for the functions investigated with Theorem 2.2.2. The size 
of Ne which contributes most to The results from the technique used in Lemma 
A.3 to obtain a nesting sequence for a given formula F. It is not known 


whether this technique can be improved. Our guess is that it cannot be.) O 


2.3 On Specker's Theorem 


In this section it will be shown how Specker's Theorem follows from 
Theorem 2.2.2 (the statement of Specker's Theorem is given in section 1,5). 

In Theorem 2.2.2 set D = {0, 1], $2 =y, a = 0 and let f be as described. 
Then by Theorem 2.2.2, we obtain that for an appropriate choice of n, we can 
find a subset X of the arguments of f with |x| =m and such that - is either 


a constant or represented by 


VF se0e oF.) 
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where :{0, it + {0, 1} and F, for 1 si <r is either a standard or reverse 
homogeneous e component over ne with the io pvaperey and constant input 
operators. The value of r is bounded as described in Theorem 2.2.2. 

We now analyze the various functions that can be represented by e-compo- 
nents with these restrictions. First note that = consists of all Boolean 
binary operators; furthermore, if f£(x,z) € ee then both x and z are free 
(because every Boolean binary operator can be represented over © in such a 
way that each variable appears only once), and thus there are no restrictions 
on the use of operators in the e-components we encounter. 

All possible graphs ['() for 9 ¢ > are shown in Fig. 2.7. The ones that 
satisfy the ft —peaperty are starred. The functions obtained by choosing a 
value for the constant input operator for the starred graphs are shown in 
Table 2.1. In general, this function is either bo ® by: Tt where TT = t (1 @ x,) 


i=1 


m 
or Co @ c,"o where o = .@ x, for some values of b b, or Cor ¢ Now, taking 


i=l “i 0’ 1 1? 
into consideration the fact that T+o = 0, and that every y: (0, ri 4-0, 1} 


can be uniquely expressed as a Boolean polynomial 


gna 
c,°M 
iso 7 ft 
where c, € {0, 1} and My is the monomial (of degree one in each variable) in 


those among Xypoeee KH, corresponding to nonzero bits of the binary representatior 
of i(see Lemma 4.5) we obtain the first part of Specker's Theorem. 

The second part of Specker's Theorem could be obtained directly at this 
point; however, we will derive a generalization of it in Example 3.1.3, and thus 


omit it here. 
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It must be pointed out that our derivation of Specker's Theorem results 
in a slightly larger bound for n; however, since no known application requires 
a specific value for the bound, this is immaterial. Specker's bound (see 


[Ho68]) is obtained from the function 


u(m,0) =m 
tatty egy PO" pe ieee cain) 

by setting Tl, (mc) = 2u(m,2c) Our bound is slightly larger due to the addi- 

tional processing implicit in the application of Proposition 2.1.7 and Lemma 

2.1.8. However, yw resembles 1, and this function by far contributes the most 

to The 3 thus, we can state that the bounds are approximately equal. 

Finally, let us note the fact that Theorem 2.2.2 allows us immediately 
to amplify Specker's Theorem. Namely, the statement of the theorem involves 
the basis consisting of all binary Boolean operators. However, the proof 
of Theorem 2.2.2 works for bases consisting of operators of an arbitrary 


number of arguments. 
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T(F) where F is an e “component 


Fig. 2.1 


‘O----G—~O-B—-O 
Y © @& 


T(F) where F is a reverse component of an e-complex 


Fig. 2.2 
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F is an SPCeC 
Fig. 2.3 
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Table of functions that can be represented by e-components 
with the 1° property (see Fig. 2.7) and constant input oper- 
ators if D= {0, }} 


Table 2.1 
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CHAPTER THREE 


APPLICATIONS OF THE GENERALIZED SPECKER THEOREM 


The principal results obtained previously [Ho68, Ho70] by the use of 


Specker's Theorem are 


3.0.1 


n 
A new proof that the Boolean function if Xs is of nonlinear length over 


1 
a 
Tl. This is accomplished as follows. First note that the restriction of the 
mod 2 sum of n variables obtained by setting certain variables to 0 is again 
a mod 2 sum (but of a smaller number of variables). Now apply Specker's 


n 
Theorem (see 1.5). Suppose iG x; is of linear length over I]. Choose n 


large enough to obtain m = 3, The theorem states that for this particular 


bases Cy = O in (1.5.1). However, it can be checked that in this case no 
choice of Co and cy will yield the mod 2 sum of three variables. A contra- 
diction. 
3.0.2 
"i n 
The function f: {0, 1} 4% {0, 1} defined by f = 1 if and only if & z= 0 
i=l 


mod 3 is of nonlinear length over X. We proceed similarly as before. Assume 


it is of linear length. Apply Specker's Theorem with n sufficiently large to 


Tt 
Of course the results of Subbotovskaya and Khrapchenko are stronger for this 
particular example. 
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obtain m = 3. If we replace X12 Xoo X in (1.5.1) once by 1, 0, 0, and 


3 
another time by 1, 1, 1, then the value of (1.5.1) remains unchanged. However, 


the value of f (with all variables except X12 Xoo X replaced by the 


3 
constant 0) is different on these two assignments. Again a contradiction. 
Both of these results were derived by Hodes and Specker in [Ho68]. 


We might note that the technique of 3.0.2 can easily be generalized to 


counting mod k where k is an arbitrary integer (see (1,5.2)). 


3.0.3 


Certain geometric predicates (see [Mi69]), in particular the connectivity 
predicate, are of nonlinear length if expressed with binary Boolean operators 
(this result was obtained by Hodes in [HO70]). We will not discuss this in 
greater detail now since this technique will be treated later in 3.2. 


In this chapter we will use Theorem 2.2.2 to generalize all these results. 


3.1 Counting mod p 


Consider the function {0, ie 4 {0, 1} 


£, (Kp siieogt) = 


0 otherwise 


then 


3.1.1 Theorem 
If p is a prime, if |p| <p, then fe is not of linear length over an 


arbitrary basis. 
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Proof 

Suppose the statement of the theorem is not true. That is, there exists 
a prime p, a finite set D such that |p| <p, a basis $ of operators on D, and 
Lite, $) S cen for some constant c. 

First note that if X is a subset of the arguments of f and Ix| =m, then 
C29 = £P. We can now apply Theorem 2.2.2. For an arbitrary m, if n is 
sufficiently large, there exists a subset X of the arguments of £P with [x] =m 
and such that aye is represented by a homogeneous en complex F (over 3° and 
with the iL property) with X as the set of lateral variables. In addition, 
since f is a Boolean function, the lateral variables of F are restricted to 
(0, 1}. 


Consider now any component F, of F and P@,) where 9, is the internal 


i 
operator of Fy. Since o, has the i property: T@,) has the general appearance 
of Fig. 3.1. Fy is determined by 0, and the constant input operator aye Now 


let m 2d and consider the sequence (s,) for 0 S j <m where 89 = a, and 


s, = 9, (11...11,a,) for j 50. Let s 


j 
j times 
that is repeated at some later point; in fact, let k(i) be the position of the 


(4) be the first element in the sequence 


first occurrence of this element. Let k(i)+£(i) be the position of the second 
occurrence of this same element. Then we shall call k(i) the prefix of F 


while £(i) will be called the period of Fy. 


Clearly, if F, is a standard (reverse) component, then 0, Ce Xpe- eX, 


-k 
11...1,a,)) where k 2 k(i) is a function of the 


i 


thieatsee) , a Ser 


number of 1's among Xyp2 XpoeeesX iy Cy peree XS) mod £(i). 
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Thus, 


If we set Y = Og a1 nek! and choose k, (ky) to 
exceed or equal the prefixes of all the reverse 
(standard) components of F, then ic 


function of the number of 1's among the variables of 


represents a 


Y mod fem(£(1),...,£(r)). 61.4) 


On the other hand, by the initial assumption, Fy 


of 1's among the variables of Y mod p; this results in a contradiction since 


is a function of the number 


d <p. 


On the basis of (3.1.1) we can obtain the following 


3.1.2 Theorem 
Let D be an arbitrary domain, $ is a certain basis, and p is an arbitrary 
integer > 1. If 3° is such that any e~component over 3° with the o=property 


and constant input operator has period one, then if is of nonlinear length 


over 4%, 
3.1.3 Example 


This is an example of a basis satisfying the hypothesis of Theorem 3.1.2. 
Consider an arbitrary domain D = {0,1,...,d-1}. Then a complete basis 
for D is YD = {min(x,y), max(x,y),0,1,...,d~1, C9 (K) 000 Oy XI} where min 
and max are defined in the usual way, 0,...,d"1 are the constants, and 
pes ifxe-t 


e, (x) = | 
0 otherwise 
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(Note that 400,13 = {A,V,0,1,-,id}; thus, a result on the nonlinearity of the 
length of a certain function over Yo, is also a result on the nonlinearity 
of the same function over Il; in particular, applying Theorem 3.1.2 to Yo, 
we obtain (2) of Specker's Theorem. ) 

tv. is interesting because it gives rise to an analog of the disjunctive 
normal form for arbitrary D: Consider the table for an arbitrary function 
f:D° 4D. Then 


ana 


f = max (M, ) 
i=0 


where M, equals 0 if the current assignment is not the en assignment and the 
h 


value of the function at the if assignment otherwise. M, is represented as 


follows 
M, = mine. cy 1) pores © a (i,n) (x dof ) 


where a(i,j) is the i component of the ha assignment. 
We claim that YD satisfies the hypothesis of Theorem 3.1.2. Note that 


5 = for all a, b € D because Vy contains all the constants. Therefore, 


* 
we will write simply 45 


Given © € ¥ with the 1°-préperty, the statement that there exists b € D 
such that the homogeneous e-component with internal operator f and b for its 
input operator has period £4 is equivalent to saying that there exists a subset 
L&D with In] = £ and o(0,z) Lis the identity (id, ) while o(1,z) Lis the 
permutation with cycle length 4 (Py). 


* 
We contend that for any L © D, (x,z) € dy and c, e €D, if o(c,z) L 
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and o(e,z) L are 1-1, then (c,z) L=o(e,z) L. Since id, # Py if £>1, 
this will establish the original claim. 

This can be proved by induction on the depth So of the distinguished 
variable z in the formula F that represents » (since there may be many such 
formulas, let F be one of the formulas where the depth of z is minimal. 

Tf bo = 1] then either F = max(F',z), or F = min(F"™,z). Assume the first 
case (the second can be argued similarly). By definition of v, F' contains 
only the variable x. If we replace x by c, F' represents a constant c' ¢ D. 
Now if c' < L, then o(c,z) L is the identity, otherwise it is not ll. 

Tf ‘ > 1, then either (x,z) = e, (0! (x,z)) where ' € M and 5! < 50 
nt < 8 (to see this, 


9 ” 
think of F). In any case ', o", and ''' satisfy the inductive hypothesis, 


% 
or p(x,z) = O*(x,0"'(x,z)) where o", pl ¢€ dy and Sgt 6 


and we are done. 


Note that Theorem 3.1.1 and 3.1.2 hold with fF replaced by the function 


Props = 
ae {0,1} given by x 


= Pp m_ ¢P 
Far = 0 mod p since gh hs {0,1} aa 


i 


3.2 Connectivity 


The connectivity predicate was already discussed in 1.6. It attacted 
considerable attention after Minsky and Papert [M169] succeeded in obtaining 
interesting results on the complexity of perceptrons that represent the 
connectivity predicate. Works that follows [Mi69] and that treat specifically 
the representation of the connectivity predicate by finite operators are, e.g., 


[HO70], [Mi71], and [Vi70]. 
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Minsky and Papert describe a circuit for computing the connectivity 
predicate of depth (of the order of) (log,n)° which on intuitive grounds 
seems minimal. This circuit translates into a formula of nonpolynomial length. 
Thus, the connectivity predicate seems to be a good benchmark for testing 
estimation methods for the complexity of functions (i.e., any appropriately 
general method which is presumed able to give estimates for length up to 
f(n) < ata should declare the connectivity predicate complex). 

Consider a set of a variables (x, 5) o 1 <i, j <n; then the connec’ 
tivity predicate is the function co! {0, iH" + {0, 1} defined as follows: 
(we will not give a formal definition since the formalization is obvious) 
Given a specific assignment to the variables, consider it as a square array 
of O's and 1's. Then ce 1 on the empty pattern (i.e., consisting of all 0's), 
or if the 1's form a connected pattern. By “connected” we mean that any two 
1's can be linked by a sequence of adjacent 1's (two 1's, corresponding to the 
variables Xj and X.g are adjacent if l4-k| + 14-2| = 1). For example, the 
pattern in Fig. 3.2 is connected. 

The general approach used here to obtain an estimate for the length of 
cy is to consider reductions of cu 

Given an arbitrary function f, a function g will be called a k-reduction 
of £ if g is obtained from f by replacing each argument of f by a function with 
at most k arguments. 

Suppose we want to prove that f. of 1 = 1,2,... arguments is of nonlinear 
length (over the basis $). Assume there exists a k-reduction 8; of f. such 


that the number of arguments m in Bn is m 2 Gn for some constant 0 <a <1. 


Assume L(£, 5%) Sc-n for some constant c. That is, there exists a formula 
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E for ff and L(F S cen. Since the length of any function of k arguments 


is bounded by L(k,$), we obtain 
L(g 8) S cL(k,5)¢n 


by making substitutions for variables in Fit 
If (8,3 is rearranged (and renumbered) in the order of increasing number 
of arguments, and all but one functions with the same number of arguments 


are deleted, then we obtain 


L(g) Sc'-m 


for some constant c'. Finally, if we can prove (e.g., by applying Theorem 
2.2.2) that Bn is nonlinear, we obtain a contradiction, and we are done. 
Hodes [Ho70] shows the nonlinearity of the length of cover X by reducing 

ce, to the function 

™m 

a pe y,) AY, 
(i.e., exactly one variable is 1) which can then be proved to be nonlinear 
using Specker's Theorem. Unfortunately, this reduction does not work for 
domains with more than two elements because this function is linear over an 


appropriate basis in such domains. However, another approach works, and we 


can state 


3.2.1 Theorem 
Regardless of the size of D and the nature of 6, cy is of nonlinear 


2 
length (i.e., L(c, 58) <cen is not true for any D, §, and constant c). 


58 


Proof 
Minsky and Papert [Mi69] succeed in reducing cto counting mod 2 by 

exhibiting a contact aetwork such that its connectivity depends on the number 

mod 2 of contact variables equal to 1, and then by simulating this network 

on the square array of variables (they call it the "retina™). We shall proceed 

similarly. 

c, is reduced to the function sP : {0, i + (0, 1} (for an appropriate 


t 


t) defined as follows: 
1 if 2 arguments are equal to 1 
» 0 otherwise 


sP, can be represented by the connectivity of a contact network sP. sP is 


shown in Fig. 3.3a. It has p contact arms for each variable Ya: The O value 
of Ys corresponds to the upward position of the corresponding arms while the 
1 value of Yy corresponds to the downward position. The contact arm of sP 
are arranged in p rows (n arms in each row). Whenever an arm for yy is in 
the upward position, it is connected to an arm for Youd in the same row; if 
the arm for Ys is in the downward position, it is connected to an arm for 


y in the next row. Thus, it may be easily checked, point A, is connected 


i+1 
t 
* Roel 


verified that in this case all the contact arms in the network are connected 


if and only if at least p among Yporerey, are 1. It may also be 


either to Ag or to Se and since these two are connected together, the whole 


network is connected. 
sP in turn can be simulated by a rectangular array RP of O's and 1's 


where certain positions are constant and others depend on the y,'8 (see 


P p 
Fig. 3.3b). The size of Ra is (3(p+1)+1).(3n+p-1), 


5g 


We now show that sP is a l-reduction of c. for some t. This is done 
by cutting RP into smaller rectangular pieces along vertical lines. The 
first piece is of length (4-1)+q where q = 3(p+1)+1 and £ will be defined 
later, the second through f-lst piece is of length (4-2).q, while the gth 
last, piece is of length between 1 and (f-1)+q. These pieces are then 
arranged into an £*q * £*q square pattern T as shown in Fig. 3.4 (the 
arrangement depends on the parity of £). Corresponding rows of adjacent 
pieces are connected by >= or C- shaped patterns of O's (in the case when 
one of the positions along the cut at the row in question is 0) or 1's. 
The unused positions of TP (corresponding to the case when the last piece 
is not of the maximal length) are replaced by 0's. 

t is set to £+q and the variable iy in c. is replaced by the correspon- 
ding position in TP (one among 0, 1, Ys OF ¥,). Obviously, the function 
obtained by this replacement is sP. If aoe 21, £ is obtained as Ix] where 


x is the positive solution of 


e231) 2 — 


We also have that n w(1/3q)t, and, thus (by the reasoning outline previously), 


P 


if c. is linear so is s*. 
t n 


With the assumption that sP is linear, we apply Theorem 2.2.2 with a = 0. 

In this way we obtain that (sP 7 where Z is a certain subset of the arguments 

of sP of size m is represented by an ey complex F (with the requisite restric- 
P 


Z 
tions) where mis arbitrary. Note that (sP)5 = 308 
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As noted in 3.1, if a sufficiently large number of variables at the 
beginning and end of the lateral sequence of F is replaced by 1, then F with 
this substitution represents a function of the number of 1's among the 
remaining variables mod the lem of a set of integers <d. The number of 
variables that have to be set of 1 is u S$ 2(d=1) (at most d-1 at each end 
of the lateral sequence). Thus we obtain a representation of the function 
oP if p2u. If p 2 2(d-1)+2, we obtain a function s, for i 22. However, 
it is clear that s; is not a function of the number of 1's mod k for any integer 


k. Thus, we have arrived at a contradiction, and, hence, co is of nonlinear 


length over any basis $, QD 


3.3. The Length of Symme Function 

As we have seen in the previous examples, Theorem 2.2.2 has been applied 
only to functions that are either symmetric or that can be reduced to symmetric 
functions. While we know of no formal statement that can be proved and that 
asserts that this indeed exhausts the applicability of Theorem 2.2.2, it 
intuitively seems probable. 

In this section we will discuss several bounds on the length of symmetric 
functions (both specific functions and all symmetric functions). Recall that 


in 1.4 we have already mentioned several such bounds (Subbotovskaya, Khrapchenko). 


t 


All of the results in this section were suggested by A. R. Meyer. 
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Does Theorem 2,2.2 (or Specker's Theorem) give us any information on the 
length of the functions investigated? Hodes and Specker do not treat this 
subject, and, in fact, the bound that can be obtained is very weak; however, 
we do mention it for the sake of completeness. 

In an application of Theorem 2.2.2 (or Specker's Theorem) to a certain 
function f£, we proceed with the assumption that L(f£,%) s cen. To apply 
Theorem 2,2.2 we must have n 2 1, (msc) where m is a sufficiently large number 
to obtain a contradiction. However, m does not depend onc. Thus, n depends 
only on c and mis assumed constant. 

Consider now c as a function of n. We ask what is the maximal value 
c for c(n) for which Theorem 2.2.2 can be applied (and a statement contradic- 
ting L(£,$) < cen obtained). c(n)*n is then a lower bound for L(f,5). Due 
to (2.2.1) ¢ grows slower than (1/4)*ht (p,n) where ht (p,n) = maximal x 


such that n 2 iexp (p,x). Then we have 
e(n)°n S$ (1/4)*ht (p,n)en (3.3.1) 


for an arbitrary constant b > 1 and for sufficiently large n. 

This bound seems unrealistically low, and it is useful to compare it 
with known bounds for length for the particular function ae over some bases 
consisting of Boolean operators (we will suppress the subscript n). 

It has already been established that - is of nonlinear length if D = 
{0O, 1} (see 3.1). We introduce the following notation: - = raat ae 
and eu stand for . x, = 0, 1, and 2 mod 3 respectively. We will repre- 


fat: 


0 
sent aa por pt by the formulas F. ri, and Fr? respectively. F is 


obtained by the following recursive relation 
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Fo (x) = Foc) A Fo¢z) V FE (y) A F2(z) V F2(Y) A F(Z) (3.3.2) 
(If X is the singleton {x}, then FOX) = x) 


L(E°(X)) = LCF CY) )4L(F° (2))4L(FE (x) )4L(F2 (Z))4L (F(X) 4 (F(Z) 


Similar identities can be obtained for F(x) and F(x). When these identities 
are used recursively, we obtain 
log,6 
2 e 
o(L(e?,8)) <n eee (3:3.3) 
an exact description how we obtain a bound of the form (3.3.3) from a recur- 
sive relation similar to (3.3.2), see [Ya54]. 
This upper bound can be further reduced by using multiargument operators. 


L R 
Let y: {0,1,2}° +%{0,1,2} be the operator =f Ys mod 3, Then 6? can be 


L=1 


represented by a formula G which uses y recursively (i.e., the arguments of 
f° are repeatedly divided by £ together with an outermost decoding operator 
{0,1,2} *{0,1}. Gis of linear length. If we use D = {0,1}, y can be en~ 


coded by two operators y' and y", and G translates into a formula such that 


1 
log, n I+ log k 


O(L(G)) = (2k) n (3.3.4) 


Thus, as £ increases, the upper bound for L(£>, 8) (where y §&) approaches 
c*n. However, the gap between this bound and (3.3.1) is still huge. But, 
the important thing to note is that any theorem that retains the same broad 
assumption (bases with an arbitrary number of operators) as Theorem 2.2.2 


cannot yield a better bound for ra than (3.3.4). 
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Another example of a function that is nonlinear in length (over all 
Boolean binary operators) by Specker's Theorem is ae However, it too has 
a relatively short representation (the previous and this example show that 
Theorem 2.2.2 is a sensitive tool for deriving the nonlinearity of functions; 
i.e., it can be used on functions that are only "slightly" nonlinear). 

A representation for ru with Boolean operators is obtained by dividing 
the arguments of ct into disjoint (nonempty) pieces Y and Z, and adding the 
bineary representations of £* cy) and ¢* (2), Let the binary representations of 
£* (x) be given by the formulas F'(X) and F"(X), obtained by the following 


recursive relations 


F™(X) = F*(Y) @ F™(Z) 


F'(X) = F'(Y) @F'(z) @ F™(Y) A F"(Z) 


(If X is a singleton F"(x) = x and F'(x) = 0) 


Consequently, 
L(F™(X)) = LCF™(Y))+L(F"(Z)) 
L(F'(X)) = L(F'(¥))+L(F! (Z)4+L(F"(¥) +L (F' (Z)) 


By choosing Y and Z always as equal as possible, we obtain 


L(F"(X)) =n 


O(L(F'(X))) = n+ log,n 


Since £* (x) is represented by F'(X) A F(X) n* logon is also a bound for 
O(L(£, &) where ©, A $, 
We now turn our attention to an upper bound for the length of all 


symmetric functions. 
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Note that a symmetric function g: D” + D where D = ~0,1,...,d-1} depends 
exclusively on Nyseee Nay where Ny is the number of variables equal to i. 


It can be represented, e.g., as 


g = max(M ) 
Meyert> Began) 


where Mi) natant) equals t+ if N, =n(i) and is GO otherwise. The number 


of combinations of n(1),...,n(d-1) is a polynomial in n -- coo -- and the 


oe 
°. 


max function can also be represented in polynomial length, regardless of the 
basis $ The latter fact is established by representing max using the two- 


argument max recursively. Thus, if M were polynomial, g would 


(1),...,n(d-1) 
also be. 

M. J. Fischer and A. R. Meyer discovered that Mi(1),..-,n(d-1) can, 
indeed, be represented in polynomial length by using a special code for 
integers described by Avizienis [Av69]. 

We will illustrate the construction on Boolean symmetric functions. It 
will be seen that if the basis of operators is appropriately chosen, the 
length of an arbitrary symmetric function is bounded above by a polynomial 
of a surprisingly low degree. 

The Avizienis code is a redundant positional representation of integers 


to an arbitrary base bs 2. We describe it for b = 3. 


An integer n is represented by all possible log,n! - tuples 


a [og,n! peue 28) 


where a, € {-2,-1,0,1,2} for 1 <i s< [og,n! and 
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The property that is exploited is that there are no long carry's in 
addition, Thus, if we want to add two Avizienis coded integers a = ee 


ay and b = beet Py> we can do it in two steps using the following 


3.3.1 Algorithm (Avizienis) 


(1) Find the carry c and intermediate sum r such that 


a, +b, = 3c, +r, 


where a by € {-2,-1,0,1,2} and c T € {-1,0,1}. 


i i’ 


(2) Compute the sum s according to 


Let us estimate the length of the formula representing any ternary place 
in the Avizienis representation of Ny for X = (X,s++0% Je 
Again let X = Y UZ and YNZ = 4. r, (&) and c, (&) can be represented 
as 
Ry (X) = pCR, (Y)4R, (Z) 564 (¥)6, 4 (2)) 


and 


G(X) = XR, (LR, (2), _5 (X50, _ 4 (2)) 


Hl and c, are O if i 51, or 1 and O respectively if 


i =1. ? and X are certain operators Pree a + {-1,0,1} which can be 


If X is a singleton, r 


obtained from the definition of Algorithm 3.3.1. Strictly speaking, the 
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domain used here is not permitted in our definition of finite functions; 


however, the difference is merely one of coding. Thus, 


L(R, (X)) = LQ, (X)4L(R, (Z)4L(G,_, (LD)FL(C,_, (2)) 


and 


L(C, (X)) = L(R, (Y))4L(R, (2) 4L(C, _, CX) FLCC, _ (2)) 


If we use these relations recursively and always make Y and Z as equal as 


possible, we obtain 


O(L(R,)), O(L(C,)) <n” 


for 1 sis Mog,nl. If D = {0,1}, we need two bits to encode Ty and Cy. 
Therefore, using certain operators 0', ep", X', and Xx" to encode P and X, 
we can encode R, (X) and Cc, (X) and combine them into a {0,1}-formula A; 


representing the ie ternary place of the Avizienis representation of Ny: 


We have 


O(L(A,)) < a? (3.3.5) 


Let there be given a positive Avizienis coded number a = Fr aa 
We desire to convert it into its binary equivalent b = arr ea Let 
US (a),.++,a,}- Then we define b,(U) = 1 if and only if the iT? bit of 


% wcaere is 1. Note that even if a is positive, b, WU) may be negative 


acy * 

i 

for some i and U. This is further discussed below. For the moment we assume 
that b, W) is always positive. We can then again compute by (ayes say) by a 


‘ wth _, 
recursive method. Let U=V UW, V MW = 4. Then b, WW) is the i- bit of 


the sum of baQ sash) and Dee be We 
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b, WU) is represented by the formula By 


B,(U) = B(B, (V),B, (W),G,_, (U)) 
.th 
where G, represents the carry from the i place; 


Go represents the constant O and §B and y are certain Boolean operators. Then 
we obtain 
5 
L(B.(U)) w © LB. (V))+L(B, (W)) 
i +=] j j 
J 
If a is the Avizienis representation of a number <n then p = [log,n| and 
q = [log nl. Thus, we obtain the following bound for L(B, (a)) 


log,log.n 
O(L(B, (a))) < (2q) rs 


log, log, n 
(2log,n) oe 


(1+log, logon) log,log,n 
logon 


wn (3.3.6) 


Note that (3.3.6) means that O(L(B, (a))) sn om) for 1s i < Mog n| where 
€?*O0asn7e, 

It has already been remarked that b(U) need not be positive. Thus 
b(U) must be treated as a signed number. If we use the 1's complement 


representation and the en=around carry technique (see, e.g., [Gr59]), addition 


can be performed as follows. Let b(U,85)=G(V)+b (W) +8, where & is either 0 
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or 1 and let 8) denote the carry from the highest position of b(U,0). Ther 
b(U) = b(U,0) if 8, = 0 and b(U,1) if 8, = 1. This means that in (3.3.4) 

‘ exp ; exp 
we obtain (4.q) for some constant £ instead of (2q) where exp has the 
value given above. 


If a, for lsjs Mlog,nl.in B, for l= i.s [log nl is replaced with 


i 


Aye we obtain formulas F, (x) representing N, in binary form. Combining 


1 
(3.3.5) and (3.3.6) we obtain 


O(L(F, (&), ))) s 2PM) 


where €(n) 70 asn 3%, 
To obtain the desired representation of an arbitrary Boolean symmetric 


function, we proceed as follows: Consider the formula S; defined inductively 


ais ia 
= 3.3.7 
Pian to! Sei” Me Seat Oral? 
Take STog. nl and replace x,. and x,, by F, and F, respectively. It is easily 


seen that STogonl with this replacement is identically 1 (this can be proved, 
e.g., by induction; for S) it is trivially true, and the general statement 
follows from (3.3.7)). 

Let there be given an arbitrary symmetric function g. It is defined by 


a subset M © {0,1,...,n} of possible values of N Each branch of length 


ni 

Mogan! in TS Tiog nl? corresponds to one value of Ny (given by the binary 

number j(Mogonl),...,4(1) where *Pogonl,j (flog nl)?***?*1,4(1) define the 

branch in question). If we remove branches of T(Sp, ) corresponding to 
og, 


M, thus obtaining the formula S', and perform the substitution defined pre- 


viously to obtain the formula p, we obtain a representation for g. 
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4+€(n) 


We have O(L(S')) <n and thus O(L(P)) <n where €*O0O asn7o, 


Thus, if a basis $ is given that contains all operators used to obtain P, 


then L(g,$) < aria. 


In Lu70 Lupanov announced a result of Khrapchenko to the effect that an 
4.93 


arbitrary symmetric function is of length sn Since the assumption were 


not made explicit, and the result itself is unavailable as of this writing, 


no exact comparison can be made with the estimate above. 
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l©,) where 9, has the 1 soreverty (only arrows labeled with 
O and 1 are drawn). I' denotes the set of nodes 0, (x x _y-- 
X50,4,) where m is as described in the text and x ree gy Xo 
may assume arbitrary values in {0, 1}. 


Fig. 3.1 
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Fig. 3.2 
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CHAPTER FOUR 


CYCLIC PERCEPTRONS 


The perceptron has already been discussed in 1.6. In the beginning 
of this chapter, we will first expand on that discussion in order to further 
motivate the study of cyclic perceptrons. 

The classical perceptron (for references on the subject see [Mi69] 
became the subject of extensive research centered around concepts such as 
pattern recognition, learning, adaptive behavior, etc. A whole myth had 
been created around it -- about its capabilities and its potential for use. 
The thing that attracted people most were its ability to learn from experience 
and its simplicity -- it combines many small decisions, the values of the 


functions Oy» into a final decision by considering their weighted sum. 


Minsky and Paper deflated this myth by showing that such a scheme has its 
inherent drawbacks. In particular, it cannot compute predicates such as 
connectivity. 

The most general intuitive basis for the result that the connectivity 
preciate cannot be represented by a perceptron is the following: First of all 
the reasoning makes sense only if the complexity of the functions 0, is limited 
in some way; if not, we can choose ©, to be the function that we desire to 
represent and then it can be represented by a perceptron trivially. Minsky and 
Papert use the order and diameter restrictions (see [Mi69]). The former is also 
used by us. 

Suppose we want to represent conmectivity. Then, if the o,'s are bounded 


in complexity (so the reasoning goes), the weighted sum is too simple a function 
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to be able to integrate all the information that is required in computing 
connectivity. 
We set out to apply the same basic reasoning to models where the inte- 
grating function is constructed out of finite operators. In particular, 
we choose addition in a finite field because of the unique representation 
property for functions in such a field (see 4.5) which makes proofs rather 
simple, and because of the purely formal resenblance to perceptrons. 
One particularly interesting aspect of using addition in a finite field as 
the integrating function is that one proof of the inability of perceptrons to 
compute connectivity is based on the reduction of connectivity to addition 
mod 2. However, this function is precisely the simplest one possible in GF(2). 
This underscores the need to make different reductions for different models 
of computation that are presumed to be incapable of computing connectivity. 
In this chapter we shall limit ourselves to Boolean functions. 


We introduce cyclic perceptrons formally: 


4,1 Definition 

cr (p*) is the finite field consisting of gk elements. & (the basis) 
is an infinite set of Boolean functions (0,1}° 4+ {0,1} such that each o € $ 
(» is the first infinite ordinal) depends on a finite number of arguments. 
Elements of $ are assumed to be ordered (in an arbitrary way). Then a (p,k)- 
perceptron (over $) is a pair P = (a,Y), where a is an W-vector such that the 
his component a, € cr (p*) and a,#0 for only finitely many values of i; 


YS GF(p*). 
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Given a function f; (o.aj- > (0,1}, we will denote the set of arguments 
on which it depends by S(f). 
Let P = (a,Y) be a (p,k)-perceptron. Then P will represent the predicate 


(Boolean function) 


fe) 


fef= a, », € YJ (4.1) 
i=0 Li 


where the value of 0; € {0,1} ¢ GF (p“). Obviously, S(f) © U S (0; ) 
LE{(j: 0, 40} 
We will indicate the function represented by a (p,k)-perceptron P as in 
(4.1), or simply [P]. 
Let us recall a concept from [Mi69]. Given a (p,k)-perceptron P = (a,Y) 


over a certain basis %, its order (ord(P)) is max (S@,) - 
1€{ j: a, 70} 


We can also introduce the order of a function. 


4,2 Definition 

The (p,k)-order of a Boolean function f over a given basis & ((p,k)-ord ,(£) 
is the smallest £ such that there exists a (p,k)-perceptron of order £ repre- 
senting f. If no such perceptron exists, the (p,k)-order of f is defined to 


be », 


Let ] be the set of all Boolean functions with finite support. Note then 
that for an arbitrary Boolean function f, (p,k)-ord(£) is finite and < S(f), 


for all primes p and arbitrary k. Also note that for an arbitrary basis % 


(p,k)-ord .(f) < (p,k)-ord, (£) (4.2) 
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We show now, as is done in [Mi69], that we can choose for the basis a 
more restricted set. Let the set of arguments of the basis functions be = = 


(XXyoeee)3 then we define the set of masks M={ A x,: S is a finite sub-~ 
1€s 


set of IN }. A convenient way of ordering M is to assign to @€M the binary 


number a ar aa where b 


defining o. 


= 1 if and only if x, appears in the conjunction 


4.3. Proposition 


Any Boolean function f can be represented by a (p,k)-perceptron over M 


for any prime p, and arbitrary k. 


The proof is the same as that of Theorem 1.5.1 in [Mi69], i.e., we util- 
ize the following correspondence between Boolean operations and operations in 


cF(p*) if the variables assume only the values 0 and lL: 


~ ° ~ - e se 
xy A Ko ~ KX ° Xs Ky Vv Xo ~ Xy + Xo ~ Xy Xoo X x 


If f is a function of n arguments, then from its disjunctive normal form, by 
using this correspondence and by multiplying out afterwards, we obtain the 


following representations for f: 


i=0 
k 
where os is the i" bit of the binary representation of i and ay € GF(p). 
Note that the mask Os (see the ordering above) is represented by the monomial 


with exponents corresponding to the binary representation of i. 
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Theorem 1.5.3 of [Mi69] also holds in our case. We state it as 


4.4 Proposition 


The following holds for an arbitrary Boolean function f, an arbitrary 


basis $, and an arbitrary integer k and prime p: 
(p,k)-ord,,(£) s (p,k)~ords (£) 


Proof 


The same as in [Mi69]. 


Note that if we take ] for the basis in Proposition 4.4, and combine it 
with (4.2), we obtain that (p,k)-perceptrons over M achieve minimal order. 


We state without proof the following well-known 


4.5 Lemma 


Every function GF(p*)® + oF (p*) can be uniquely represented as a polynomial 
k : 
in n variables over cr (p*) that is at most of degree p -1 in each variable. 


(see, e.g, [La67].) 


It has already been noted that we will be interested in whether a function 


can be represented by a (p,k)*perceptron with a limitation on its order. For 


this we need the following 


4.6 Definition 


A sequence of Boolean functions Ei ofosees of 1,2,... arguments is of 


+ s 2 s : 
finite (p,k)-order (over a given basis $) if there exists a finite r such 


"pounaed would be a better word, but we conform to the terminology of [Mi69] 
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that for alli (p,k)~ords (£,) =r. 
Let there be given a (p,k)-perceptron (a,Y). If Y = (Yyoeees¥ ds then, 
; k 
recalling that GF(p’) is a vector space of dimension k over GF(p), and desig- 
F .th 
nating the j component of a, by ayy (similarly for Yn € Y), we have 
& m k 


[ 2 cao €¥] =e Vv CA fa. O. = ¥ J] (4.3) 
iso tt hel j=l iso tJ E OhI 


We can restrict the diversity of perceptrons we are dealing with by noting 


4.7. Proposition 


Let $ be a basis closed under conjunction (i.e., 0, bE F > PAY E 4). 
If a Boolean function f is of finite (p,k)-order over $, then it is of finite 


(p,l)-order (but the order may change). 


Proof 
We have f = [(a,Y)] where (a,Y) is a (p,k)-perceptron. Suppose ly| =m 


and the (p,k)-order of f is £. From G.3) we have 


k 
A [€ © a,,°o, = ] (4.4) 


ij ot ~ "hj 


where 445° Yay € GF(p). 

By Lemma 4.5, we know that for all a € GF(p) there always exists a poly- 
nomial P(x) over GF(p) of degree p-1 which takes on the value of lif x =a 
and is 0 otherwise (the degree follows from the number of zeros of the poly- 


nomial). Thus substituting the Boolean operations with the field operations 


introduced in the proof of Proposition 4.3, we obtain from (4.4) 
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k © k © 
f=Q( Il P C2. sa oe Ge bes. ID P ( = a,,+,)) (4.5) 
j=l Yj iso Ht j=l %my a-o 4 i 


where Q(x) +60 5%) is the polynomial (of degree m) that represents the Boolean 
m 

function V x Each P is of degree p-1. Hence f can be expressed as 
h=1 ij 


a polynomial in the 0; 's of degree < m*(p-1). Obviously, o for j > 1 can be 
replaced by °, since it assumes only the values 0 and 1. Also, *} represents 
the function oA¥ and |S A ¥)] < [S(m)| + [S(¥)|); thus, if the basis is closed 
under conjunction (as, e.g., Mor M), (4.5) describes a (p,l)-perceptron for f 


of order < m:(p-1)°4. 0 


Remark 
Incidentally, this proof also shows that we can assume the cardinality 
of Y to be l. 
Since we shall subsequently be concerned only in whether the order of cer=- 
tain functions is finite or not, we will be able to limit ourselves to (p,1l)- 
perceptrons. For convenience, we will write simply "p-perceptrons™, Also, we 
will be only concerned in whether there exists a basis over which a function is 
of finite order. This is equivalent to whether a function is of finite order 


over M. 


t 
Q(x, o+-+ x) is obtained by using y, V V5 Va Pg Ny recursively; 


Lee, Q(K, +++ %) = Q(X) pee 9X apt x,t Hy Vy eee Ke If Q is a poly- 
nomial over GF(2), then Q=@ MI y where the sum ranges over all nonempty 


y€s 
subsets S & (xy s2+0.%)- 
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We first turn our attention to the case when p = 2. Instead of "2-percep= 
tron", we will say "Boolean perceptron". 

From Lemma 4.5 we conclude that every Boolean function can be uniquely 
represented as a polynomial over GF(2) that is at most of degree one in each 
variable (a Boolean polynomial). 

Noting that the terms of a Boolean polynomial represent marks, we conclude 
that every Boolean function f has a unique representation as a Boolean perceptron 
over M. Furthermore, by Proposition 4.4, this representation is a minimal order 
representation for f. Note then that 2-ord, (£) corresponds to the degree of 
the Boolean polynomial for f. 

This unique representation property allows us to establish the minimal 
order of certain interesting predicates very easily. As in 3.2, we are again 
interested only in functions £0.15 + {0,1} that are interpreted as functions 
of nxn patterns of 0's and 1's. In particular, we are interested in the Boolean 
function of ar variables cy (introduced in 3.2) and @n kk (the Euler number of 
a pattern of 1's on a square array of 0's and 1's is equal to k). It is well 
known (see, for example [Mi69]) that the Euler number of a planar figure is 
the difference between the number of its components and the number of its holes. 
If we use the notion of connectivity introduced in 3.2, then the Euler number 


of the pattern in Fig. 4.l isl. 


4.8 Theorem 
The connectivity predicate is not of finite 2-order over M (hence, over 


any basis). 
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Proof 

We use the One-in-a-box construction introduced in [Mi69]. Before pro- 
ceeding, however, we must define certain auxiliary predicates. n, the size 
of the pattern is assume odd (henceforth, we will suppress the subscript n 
in the notation for functions). The variables representing positions in the 


square array will, as usual, be denoted by x,. for 1 <i, { Sn. Then we define 


ij 


r= (4 A X19 Kee AN Xin) A 


and 
s = (X51 Vv Xo9 Vieee V Xo) A 
4.2 Vv ee V x) A eee A 


V ys 


Vio s Vv 
*n-1,2 *n-1,n 


i.e., r is 1 only on patterns with odd rows consisting exclusively of 1's, 
and s is 1 only on patterns where each even row has at least one 1 (the One- 


in-a-box predicate). Then, 
rAcezraAs (4.6) 


(c is the connectivity predicate). 
Now, for arbitrary functionsf, g, h,if h = f A g, then 2-ord,,(h) < 2-ord,, 


(£)+2-ord,,(g); i.e., 


2-ord,, (g) 2 2-ord,,(h)-2-ord,,(£) (4.7) 
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Replacing h by r As, f£ by r, and g by c we obtain 


2-ord (c) 2 2-ord,, (r A s)-2-ord, (x) (4.8) 
We have 2-ord,(r) = Bei *n; 2-ord (h) = n=3 *n (recall the Boolean 
M a2 ie a 2 : = =e 


m 

polynomial for Ae x, described in the footnote on p. 80 ) 2-ord, (r As) = 
L1= 

n(n-1) (because ;the Boolean polynomial representations of r and s have no 

variables in common). Using this we obtain from (4.8) 2-ord,,(c) 2 nne3) ; 


i.e., the 2-order over M of the connectivity predicate is not finite. oO 


We next establish 


4.9 Theorem 
The predicate "the Euler number of a pattern equals k" is not of finite 


2-order. 


Proof 

We again consider the case when M is the basis. The general case follows 
from Proposition 4.4. mn is the size of the pattern. We need to consider a 
subset T&S = (x, 5° i+j even} (note that all points of S are disconnected 
from each other, in the sense we use this word). |r| = t will be determined 
subsequently. 

We define the following predicates 

pr = 1 if and only if all points of T are 0; ice., 


pr = I (1 @x) 
x€T 


q = 1 if and only if k points of T are 1; i.e., 


84 


q= 8 I x- HT (1 @y) 
x€U yeT-U 
where the sum ranges over all possible subsets U & T with lu| =k. When the 
expression for q is multiplied out each term produces exactly one term of the 
form II x and thus the above Boolean polynomial is of degree t if and only 


x€T 
if the number of terms in the sum is odd. The number of terms is q- But 


L 
2°~ t 
( 1) is odd for all 0 Sk = 2 <j] and all &. Thus if t = 21, Oskst 
k 
then 2-ord(q) = t. Also, 2-ord(pr) = [| = aot: 


Recalling once again the e, is the difference between the number of compo- 


k 


nents of a figure and the number of holes, we have the relationship 
pr A e, = pr Aq 

Again using (4.7) with g = ey h = pr Aq, £ = pr we obtain 
2-ord,,(e,) zn? |t| = of 


No matter how large we choose £4, we can find an n such that we can obtain 
a set T with |r| = 24, Thus, the Euler predicate is not of finite order 


over M. q 


Theorems 4.8 and 4.9 can be extended to p-perceptrons for arbitrary 
p. The generalization will only be indicated for Theorem 4.8. 


£ 
"proof: First show that ee is even for all 4 and all k#0, 2". This is done 


by induction. Now observe that due to Q@ = Cc) + es), and the fact that 
4 L 4 
Ss > is odd, c ae is also odd (for otherwise e }) would not be even). We 


can continue this way and establish the claim. 
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The obvious difficulty is that Boolean functions do not have a unique 
representation as polynomials over GF(p) for py 2. Specifically, in the 
case of Boolean perceptrons over M, we were able to reduce the problem of 
the order of connectivity to the order of r and s (see above). The orders 
of these predicates (equal to the degrees of the corresponding Boolean poly- 
nomials) were easily computed due to Lemma 4.5. 

Suppose c is of finite p-order over M (we have already remarked that this 
brings no loss of generality) for some p > 2. Due to Proposition 4.7 we can 
assume that we have an expression of the form of (4.5) for co When multiplied 


out, we obtain 


c= & a,m, mod p (4.9) 


h 


where m, is the monomial representing the at mask. is of degree one in 


| 
each variable, and the values of the variables are restricted to {0,1}. Since 
the perceptron from which we obtained (4.9) is finite order, we can assume that 
the degree of (4.9) is < &. 

We can now extend c, to the domain GF)p) (i.e., to square patterns A = 


{b,,}> 1<i, j <n, and b, € GF(p)) by defining co) (A) = 1 if and only if 


j 
ce (£(A)) = 1 where f(b, 4}) = fe, ,) such that Cy, = 1 if bs; = 1, otherwise 
Cry = 0. We have from (4.9) 
co 
ct a Rae a, eM; mod p (4.10) 


where m! is obtained from m, by replacing each variable x, by Py (5) such that 
i 
P, (x,) = 1, otherwise Py &,) = 0 (see the proof of Proposition 4.7 how to ob- 
J 


tain P, (x,))- Now we have a total function c’ over GF(p), and thus its 
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polynomial representation obtained by multiplying out (4.10) must be unique. 


Since the degree of (4.9) is < 4, 
the degree of the polynomial (4.10) < (p-1)4 (4.11) 


Another estimate of the degree of the polynomial for co is obtained using 
the predicates r' and s' obtained from r and s of Theorem 4.8 similarly as 
ch was obtained from cj+ ‘The polynomial P_, representing r', is obtained from 
the polynomial representing r by substituting each variable x with P, (x). 
Similarly, for the polynomial representation Po of s'. The degrees of Pe! 


- atl n+(p-1) and = a3 n(p-1) respectively 


and Pat are then found to be 
(2-ord,,(r) and 2-ord,,(s) multiplied by deg(P,)). Since the analogs of (4.6) 
ad (4.8) again hold, we obtain that deg(P..) is not bounded, contradicting 
(4.11), and thus also the finite order of c. 

Note that the obvious generalization, i.e., if a function is of finite 
2-order, then it is also of finite p-order (over M), is not true: Consider 
the Boolean function @ Xs We will investigate the degree of the polynomial 
representation of etait in the same way from ® as c' was from c above). 
If a p-perceptron over M of finite order exists for @, then we obtain a poly- 
nomial representation of bounded degree for @' similarly as (4.10) was 
obtained from (4.9). On the other hand we have the following representation 
for & 

xy a X, a (1ex,) mod p (4.12) 
i€s igs 
where S ranges over all subsets of {1,...,n} of even size. When multiplied 


out, each term produces exactly one monomial of the form Xie The number 


hows 


i=1 


of such monomials appearing in, the deyplpped, form of (4.12) is 


6f1900000 
ofrrono6ca 
ub Cat a 
ai" 8 oofor a 
18 CC00TUI0 
Se 206000 
(use the ddentity (f) = ("796 GUY 3.0 Mace this mmber is not 0 


mod p, (4.12) yields 9 acpetneir Tn MAL Heer = for % If x, 1s 
replaced by P, &,) we obtain « (yndqug) ypolynemdal representation for 
@' of degree n° (p-1), contradicting the existence of a finite order 


p-perceptron over M for & 


ve 
Sa 
i Aas cay. x 
i 
8 
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CHAPTER FIVE 


PATTERN COUNTING MACHINES 


In this chapter we shall permit ourselves a certain degree of informality. 

We are again concerned with the power of machines that combine a large 
number of "local" computations through an integrating function. Only this 
time we shall not be limited to functions that can be represented as a combin- 
ation of finite operators. 

This class of machines again operates on square patterns of O's and 1's. 
The operation is divided into two phases: In Phase I the pattern is scanned 
with a square "window" of a certain size. Each time a nonzero pattern appears 
in the window, we take note of it (there is a finite number of nonzero patterns 
since the window is of finite size). At the end of the scan we have a count 
of the various patterns,and we are then allowed to utilize this data in Phase 
II which consists of computing the value of a partial recursive function for 
this data. The formalization of this model is obvious and we omit it. 

What can such a machine do? Clearly the computation of this machine is 
divided into a local phase and a global phase, so that it fits into the broad 
class of problems considered in [M169] and Chapter Four. 

Note that the boundedness of the window size is essential. If we insis~ 
ted only that the window contain a given number of points, but otherwise 
allowed it to be of any shape with arbitrary distances between its points, 
then Phase II could reconstruct the whole figure as was observed already in 
[Mi69]. 


We again inquire whether these machines can recognize the familiar 
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topological predicates connectivity and Euler number. 


5.1 Theorem 

Pattern Counting Machines (PCM) cannot recognize the connectivity pred- 
icate. 
Proof 

We need only exhibit two patterns, one connected and the other discon- 
nected, with the same pattern spectrum. In this case, no algorithm of Phase 
II could establish the difference between them. 

Two such patterns are given in Fig. 5.1. 

Specifically, these patterns are equivalent under windows of size 2x2. 
However, it is easily seen that increasing the dimensions of the patterns 
in Fig. 5.1 linearly by a factor of k makes them equivalent under windows 
of size up to k + 1. We can arrive at this conclusion by setting up a 1-1 
map between occurrences of the same pattern in the window in the two pat- 


terns 


5.2 Theorem 
PCM's can compute the Euler number. 
Proof 
It is shown in [Mi69] how to compute the Euler number from the spectrum 


of patterns of the shape 


oo fo | 
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Before proceeding, we need a notion of continuous deformation. Pattern 
B can be obtained from pattern A by continuous deformation if B arises from 
A by a sequence of additions or deletions of 1's of the following kind: Let 
us fix attention on a 3x3 square with the central position in the place of 
the 1 being added (deleted). For simplicity assume that the boundary positions 
are always 0. Each position of the periphery of the square which has a 1 in 
it is either connected or disconnected to another 1 on the periphery (not 
necessarily by a path in the square). This set of connections may be described 


by a symmetric 8x8 0-1 valued connection matrix, i.e., a,, = 1 if and only if 


ij 
the i and rigg positions on the periphery have 1's and are connected. The 
proposed addition (deletion) is permitted only if (1) the connection matrix 
remains unchanged as a result of it, and (2) there is a 1 adjacent to the 
proposed addition (deletion). 

Any predicate whose value remains unchanged if the pattern A is replaced 
by B, obtained from A by continuous deformation, is called a topological 
predicate. We assert without proof that connectivity and Euler number are 
topological predicates. The reader is warned, however, that there is a pitfall 
in proving this fact for the Euler number predicate. The number of holes in 
Fig. 5.2 should be one, not two (i.e., O's are connected diagonally in addition 
to their usual connections). This is discussed more fully in [My71]. However, 


if the holes are sufficiently large (so that all the O's in them are connected 


in the usual way) this difficulty is not encountered. 


5.3 Theorem 
Any topological predicate recognized by a PCM must be a function of the 


Euler number, 
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Proof 
We will have established the theorem if we succeed in showing that, 
given any PCM P computing a topological predicate, then for two figures 
X, and X, with EULER(X,) = EULER(X,),we also have P(X) = P(X). 
In [Mi69] it is shown that for every figure X there exists an "Euler 
canonical” figure C(X) such that EULER(X) = EULER(C(X)); and if for two 


figures X EULER (X, ) = EULER (X,) , then C(X,) = c(X,). If the Euler 


1? *9s 
number of X is n > 0, then C(X) consists of n components without holes. If 
the Euler number of X is n S$ 0, then C(X) consists of 1 component with-ml 
holes. 

We will show that we can deform any figure X into C(X) without changing 
the value of P. 

The deformations available to us are: 

(1) Continuous deformation. If we subject X to this kind of deformation, 
then P(X) remains unchanged because it computes a topological predicate. 

(2) Deformations that leave the pattern spectrum unaltered. By defini- 
tion of PCM's, 

As a consequence, we have 

(3) Removal of components inside holes. To accomplish this without 
changing the value of P(X), we first apply (1) until the window cannot scan 
simultaneously an interior component and the wall of the hole in which it 
resides. Then it is obvious that the pattern spectrum will remain unchanged 


if we remove the component from inside the hole. After this we can apply 


(1) in the reverse direction. 
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If we are given two figures X and X, with same pattern spectra, then 
we can add equally shaped holes to the figures in such a way that the pattern 
spectra remain the same. The holes have only to be placed in such a way that 
the window cannot scan any other boundary while scanning the newly introduced 
hole. We can then repeat this to add any number of holes. 

Specifically, given two figures of the shape of A and B in Fig. 5.1, we 
can add holes in this way and still have the same pattern spectra. For example, 
C and D in Fig. 5.3 have the same pattern spectra for a sufficiently small 
window size. Note that given any two components, one of which has a hole, 
we may apply deformation (1) to obtain a figure proportional in dimensions 
to C and they apply (2) to obtain D. We call this sequence of deformations 
“cancelling a hole and a component", 

We deform X into C(X) by cancelling as many holes and components as 

possible. We first apply (3) until no component remains within a hole. 
Then we may either have a hole and a component not containing this hole, or 
not. In the latter case, we are done. In the former, we select a hole and 
a component not containing it and cancel them. After this we are left with 
one less component and hole. We repeat this until we arrive at C(X). 

We can summarize the results on the recognition of topological predicates 
contained in [Mi69], Chapter Four, and in the present section in the following 


table 
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Recognition 
of 
Predicates 


Classical 
Perceptrons 


Cyclic 
Perceptrons 


Connectivity 


Other Functions of Functions of 
Topological Euler number Euler number 
Predicates only only 


Thus, all these results support the conjecture expressed in [Mi69] that 
no "local-global" computer can recognize connectivity. 

It appears, however, that all models are extremely sensitive to altera- 
tions. We have already mentioned how PCM's can be converted into universal 
machines with the removal of the restriction on the size of the window. 


A. Re Meyer noticed that ordinary perceptrons may be modified to recognize 


feo] 
any Boolean function with order one. Instead of & a, O, 2 0 consider 
= i=0 
by a, O; € Y for some subset of integers Y. Now we can choose the coeffi- 
i=0 


cients as in such a way that the sums of the coefficients in no two subsets 


of the set K of all coefficients is the same, ie., 


Y&YSRK)[ Eas fa 2k = ¥] 


a, €X a, € Y 


We can define the coefficients inductively, i.e., choose a to be greater 
n-1 

than zu ay: This is in the spirit of stratification (see [Mi69]). Now 
i=l 


notice that if % is the set of mesks of order 1, then a Boolean function 9 


Oa © @o 
o° © 
oOo Om Oo 
CO nog tre Ge mq 
J 
OO Ont o 
ooobteo 
oo oea 


is simply 


of integerg pepreeqngigg the sums of 


9000000 


60006000 


belonging to §%. 


a 
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oooo0coceo 
ooonrooo 
ooornroo;o 
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oooroo°o 
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ooorooco 
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Two figures with the same 2x2 pattern spectra 


Sek 


Fig. 
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The number of holes in this pattern is one 


Fig. 5.2 


Cancelling a hole and a component 


Fig. 5.3 
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APPENDIX A 


CERTAIN PROPERTIES OF SHORT FORMULAS 


The purpose of this appendix is to modify certain results of [Ho68] in 
the light of our different requirements. Our goal is Lemma A.9 which is used 
directly in the proof of Theorem 2.2.2. We prove it by way of a series of 
intermediate results, none of which are used elsewhere. 

In what follows we would frequently use the phrase "F is a formula in 
n variables over @, and such that no variable occurs more than k times". 

This will be abbreviated to "F is a ($,n,k)-formula". If any of the para- 
meters is not present, we will replace it by *. For example, "F is a (6,*,k)- 
formula" and "F is a (3,n,*)-formula™ mean "F is a formula over %, and such 
that no variable appears more than k times™ and "F is a formula in n vari- 


ables over $™ respectively. 


A.l Definition 


Let there be given the sequence of formulas G = (Gy Ky s2)a-00 Gy 


(K-122) 6, &))- If 1 <i < p-l, then G, contains the distinguished 


i 


variable z, occurring only once. X, for 1 <i <p is nonempty and is 


either a singleton or © |) X,. Let F be an arbitrary formula, and 

jt 
Gs Gy Ky Gy Ry 500+ Gy KG (K++ If F = G, then G is a nesting 
sequence of length p for F. If, in addition, the total number of occur- 


rences of any variable (except z) in G is < the corresponding number in F, 


then G is a proper nesting sequence for F. 
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A.2 Remark 

Let G and @ be as described in Definition A.1l. Furthermore, let all 
X, for 1 Si Sp be singletons and distinct. Then G is equivalent to an 
ge Ne le Also, suppose Xx, is arbitrary and G, is a formula over &. 
Now replace all variables except possibly one in G, for 1S i S p-1 by the 


constant ae Let the set of variables that have not been touched be Y. 


Y 
Then GC. is equivalent to an e-component over 37. 


Let F be an arbitrary formula over $, X F S(F), and a € D; then we 
would like to obtain a formula over §* with the following properties: 
(1) G = F. (2) S(G) = X, and (3) the number of occurrences of any vari- 
able of X in G is $ the corresponding number in F. G can be obtained 
by a straightforward replacement of operators in F such that the variable 
symbols that are replaced with a in forming FS (and subformulas of F 
where S(F) consists entirely of such variable symbols) are removed, and 
the remaining operators are changed to preserve equivalence with F 
More precisely, if OCF, .+++5F,) is a subformula of F, then if S(F,) tx 
for all i, remains the same; if S(F,) Cc X and S@,) ¢ X for 3#1, then 


© is replaced with O(Ky see 9X 9X) where all variables of 


geat® ge ygae** 
Fy have been replaced with a (if there are more such indices i, we proceed 
in the obvious way); and if S(F,)CX for all i, then © is eliminated. This 


transformation will be called normalization and G will be denoted by norm(Fs). 
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A.3 Lemma 
If F is a ($,n,*)-formula, then for any p, q 21 and a € D, if 
n 2 1, (p,q), there exists a subset X © S(F) such that either 


(1) [xl = q and F. is equivalent to a PC of the formulas Fyasee oF) 


where r S$ eee and Fy for 1S i <r is a formula over 54 such that each element 


of X occurs in at least two among F Fs and the total number of 


wer 


occurrences of any x ¢ X in F oF is S the number of occurrences of x 


pr 
in F; or 
(2) Ixl is arbitrary and F. has a proper nesting sequence G = (Groves 


Gna oe where G, for 1 Si <p is a formula over 57 
Proof 

Assume there is nO xX © §(F) such that FS is as described in (1) of 
the statement of the lemma. 

We will describe a (proper) nesting sequence extraction procedure 
(NSE) whose inputs will be a formula H over 6° and a set of variables 
Y. The output of NSE will be two formulas H'(Z,z) and H™ over $* such 
that Z is either a singleton or & Y; furthermore, HL = H'(Z,H™) for some 
U © S(H). 

G will be obtained by the repeated use of NSE. Initially, the input 


of NSE will be F = F, and ¢. In the first application of NSE, the output 


0 


will be G) and Fy (Fy is an intermediate formula whose significance will 


be describe immediately). In general, the a application of NSE will 
receive the input Fe and |! x and yield as output GC, and Fie We will 


1 ae 
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show that if n 2 Te (P24), we can apply NSE p-1 times and end up with F-2 
from which G. is obtained as will be described below. 

Description of NSE. The input to NSE is as describe above. Then 
we can distinquish two cases: 

Case I. L(H) = 1. In this case we cannot apply NSE, and the output 
is undefined. 

Case II. L(H) 31. In this case we can assume that H has no unary 
operators; for suppose there exists a subformula J of H such that J = o(t( 
Jyoeeesd)) where J; for 1s i <r is either a variable symbol or another 
subformula of H. In this case Ov = p € 6° and we can replace J by the 
equivalent formula PCT, see0od). Similarly, if J = OCT, 000 WS) se00sTy)s 
we can eliminate } because OKs eee VK, Dy 000%) = P (Ky see Xs vee eX) 
€ 34 (thus, if a unary operator of H corresponds to an internal node of 
T(H), we can eliminate it by either of these two means; on the other hand, 
if a unary operator of H corresponds either to the root node or to a node 
next to a terminal node of T(H), then we can use only one of the two methods 
described). Now choose i' such that SCH y1) is maximal among S(H ,) for 
1s isr. Since support is defined only for formulas, SCH y 1) may be 
undefined if all arguments of the outermost operator of H are variable 
symbols. In this case replace one of them by the identity operator 
which is possible since id ¢€ 8°, Gonsider H/H ys = K(Z,z). Again two 


cases can arise: 
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Case Ila. YMZ=@¢. Choose any variable of Z, e.g., x, and let 
H' = norm(x!*271, z is a distinquished argument (hence x is free). 
In this case set V = S(H 72 Uf{x}. The significance of V will be 


seen immediately. 


Uf 


Case IIb. YN Z#d¢. H' = norm(K* ) z is again a distinguished 


argument V = S(H pnd ies 
In both cases H™ = norm((H gO.) 


Analysis of NSE. Let Is(H) | =m, and let us estimate Iscut) |. 


m 
n e 
max 
of applications of NSE to a formula F, and F does not satisfy (1) of the 


Obviously, ISH iol 2 In the case that H results from a chain 
statement of the lemma, then we claim that in Cases IIa and b less than 
q variables are set to a in H ute Suppose this is not true. Let the 
set of variables that is set to a on this occasion be W. Then W & Z-{x} 
(Case IIa), or WS Z-Y¥ (Case IIb). In any case consider ae This is 
W W 
eee ease th 
(1) a? Hs ya where i(1), ,i(s) are the 


indices corresponding to the subformulas H j where all variables have 


equivalent to PH i 


not been replaced by a (if in H , all variables have been replaced by a, 


k 
then it is absorbed into ~). But then o(norm((H 4 (yy a) see norm (H 5 (g))a)) 


satisfies (1) of the statement of the lemma. A contradiction. 


m 


n 
max 


Thus, Is(Ht)| 2 =qtl 


Hence, if we define 
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T(1,q) = 1 


Tg (pt1.q) = (% (p,q)+q-1)> eee 


l.e., Poly 
A p-l. max pid 
1 (P.q) =a eat Dy tar )) 
max 


for p,q 21 (and if F does not satisfy (1)), we will be able to apply NSE 


p-l times and obtain Foe G, can then be obtained as follows: If SF) 
al S(G.) = @ for 1 <i S p-1, then choose any variable y ¢ re ai 
obtain Ce from Ceol by normalization; otherwise, denoting U S(G,) 
by U, obtain Eel from @ A), by normalization. It can be diated that 


G, for 1 si <p satisfy the conditions of (2) of the statement of the lemma. 


im 


Consider a sequence of (nonempty) sets X, for 1S i S p such that 


i 


X, is either a singleton, or is included in VU X,. 
: jaa J 
a sequence of sets a normal sequence (of length p). Note that the sequence 


We will call such 


X ue in Definition A.1 is a normal seqtence. Then 


yer? 


A-4 Lemma 


Let X 


peersak, be a normal sequence of sets with the additional pro- 


perty that each element of ey Xx. appears in at most k elements of the 
i=l 
sequence. Then if p 2(k+1)™, there exists a subset Y S Pi X, and an 
i=l 
increasing sequence of indices i(1),i(2),...,i(q) such that (1) q 2m, 
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(2) 11) = 1, (3) XG) NY is a singleton for 1 < j < q, (4) Xy Ny=¢ 
if 24s i(q) and 4 #1(j) for 1 < 4 <q, and (5) if x € Y, jy < j, < jy 


and x € X x € xX, ? then also x € X 


iG)? Gy iGy)" 


Proof 
(this is a direct translation of the proof of Lemma 2 of [Ho68] into 


p 
our terminology). Let X, = (x1 Xp re0-)e Without loss of generality 
boy 


i 
i 
assume that x) € xX): If m= 1, set Y = (x,J; i(1) = 1, and conditions 


1-5 are satisfied. For the inductive step two cases are distinguished. 
Case I. x, occurs in none of the sets Xi 2<j6 (e171 +1. 


Setting r = (1) hg , the sequence X ok, is normal and each element 


gers 
r 


occurs in at most k of the Xs 2255S ro TE 7 SU] x, and the sequence 
j=2 
j(1),.--,j(q-1) are obtained by the inductive hypothesis, then (x) UzZz=¥Y 


and i(1) = 1, i(2) = j(1),...,i(q) = j(q-1) satisfy conditions 1-5. 
Case II. Assume that x, occurs in some Xs 2<js (e174 and 
let h be the smallest such number j. Furthermore, let V be the set of 


elements different from x,, and occurring in XoseeesX as Delete the 


1 
elements of V from Xp oK vee kX 


p? 


and delete those among RiAgst aX, 


that remain empty. Let the resulting sequence be Y a3 The length 


qc. 
m-1 
of the sequence CX Xp Kage eX) is at least p-(k+1) “+1. 
There are less than enh ili distinct variables in Xoyeee aX yo 


each one occurring in at most k-1l of the formulas aaa aaa 


Therefore, 
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be pac) ee eta ae 

r= (k+l) 4h 
The sequence Yooree oY) is normal and its length is at least Gao 
x, occurs in Y,. Let Z2¢ ty Y, and the sequence j(1) = 2, j(2),..., 
(q-1) be obtained ete the inductive hypothesis for Yoreero¥,s 


Then Z and i(1) = 1, i(2) = j(1),...,i(q) = j(q-1) (where q 2m), satisfy 


conditions 1-5. 


Let there be given a (*,*,k)-formula F with the proper nesting 
sequence G = AS oes? such that G, is a formula over &. As has 
already been remarked above, Riek (see Definition A.1) is a normal 
sequence of sets. 

If p2 (kt+1)™, then by Lemma A.4 there exists a set Y & Pi X, and 
q indices i(j) for 1 < j Sq such that conditions 1-5 hold. eee that 
if m = kt, then ly| 2 t since no variable appears more than k times 
in G (Gis proper). In particular, consider only Z = (Xp 00+ 9X) SY 
where KyporeesX, are numbered in the order of their appearance in G. 

Note that due to condition 5 of Lemma A.4, if x, y €Yand y follows x 
in G, then x cannot appear again after y in G. Let G be as deined in 
Definition A.1. Then we will let the reader convince himself that Ge 
(hence also F) is equivalent to K(Z,G') where K(Z,z) is an e, component 


a. . . : . a 
over $ with input variable z, and G' is a certain formula over $ such 


that each variable of G' occurs at most k-1 times. 
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Note that in this case we do not know the size of S(G'). This 
can be remedied in the following way: There are two cases; either 
ls¢e"y] 21/2+t, or not. In the first case perform an a-merger on K 
with basis S(G'), after which we obtain an SC of an e=component K' 
of length 2 1/2*t and a formula G” (through the input variable) such 
that S(G™) equals the set of lateral variables of K'; in the second 
case perform an a-merger on K(Z,G') with basis Z-S(G') in which case we 
obtain an e-component K' of length 2 1/2*t with a constant input operator. 


We summarize the preceding in the following 


A.5__Lemma 

Let there be given a (*,*,k)-formula F with a proper nesting sequence 
of length p 2 1 composed of formulas over $. Then if p 2 (ij *, there 
exists a set 2 © S(F), [F | 2t, and FY is either equivalent to an SC of 
an e, “component K over 8° and a formula G over 8° such that S(G) is the 
set of lateral variables of K, and no variable of G occurs more than 


k-1 times in G; or to an e, component K over 37 with constant input 


Operator. 


Let there be given a PC F of the formulas F peeeoF where r = nh 


1 


such that Is(F)| = q and each variable appears in at least two among 


ax 


F,,...,F_. (i.e., a situation as described in (1) of the statement 
1 r 
of Lemma A.3). We are interested in obtaining a (nonempty) subset 


X © S(F) such that when the variables outside of X have been replaced 
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by the constant a, |s(norm(F, )*))| for those Fy where not all variables 
heve been replaced with a is equal or larger than a predetermined number 
t (as large as possible). 

We could solve the problem as follows: Each variable of S(F) appears 


in a certain subset of the formulas F Fue The number of such 


yeorr'? 


max 
) 


° r F P 
subsets is 2° (in general, < 2 ; thus, we are sure to find a subset X 


with |x| 2 — such that all elements of X appear in the same subset 


2 max 

of FroseeoF 

However, we can improve this number. Let us construct the occurrence 
table of F. The table consists of rows corresponding to elements of S(F), 
and of columns corresponding to Fy for 1 si<r. The entry aaj is l 
Lf xy occurs in ey and 0 otherwise. We will try to extract a subset 
X © S(F) such that either all variables of Fy are replaced by a, or 
S(norm(F,)2)) contains 2 elements (t will be determined later). 

If all columns in the occurrence table contain 2 t 1's, we are 
done and X = S(F). Suppose not. Let the column j contain ¢ t L's. 
Delete all rows corresponding to the 1's in column j and column j 
itself. Let the set of variables corresponding to the remaining rows 
be X- Consider the remainder of the occurrence table (i.e., minus 
the deleted rows and column); and again look for the column with < 1's. 
If it does not exist, we are done and X = Xi. If such a column exists 
continue. Now two things can happen. Either at some point we end up 


with a certain subset of columns, all of which contain = t 1's, or we 


end up with two columns that both contain < t 1's. We shall see that by 
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an appropriate choice of t, the latter case cannot happen. The number 

of 1's in the whole table 2 2q (each variable occurs in at least two 
formulas). The smallest number of 1's remaining after all but two columns 
have been deleted 2 2q-m where m is largest possible number of 1's that 
can be deleted in the course of this procedure. m = (t-1)-(rtr-l+r-2+... 
+3) = (t«1)° fetnirce) (this corresponds to the case when each deleted 
row contains only 1's and at each stage t-1 rows are deleted). If, after 
the table is reduced to two columns, both columns are to contain 2 t 1's 


(both have to contain the same number of 1's since each variable occurs 


in at least two formulas), then 


2q-m > + 
2 
or since r Sn 
max 
4qte " 4 
ts ed where c (hoax t?) nar 2) 


For large Tax this is better than the previous bound. This result 


can be summarized in 


A.6 Lemma 
Let there be given a PC F of the formulas FroeeeoF over $ where 
rs A aie such that Is(F)| = q and each variable appears in at least 


two among F Fe Then if 


yer’? 
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q = fetes where t 21 


we can find a subset X S S(F) such that Fr is equivalent to a PC of 


the formulas Gree Gy over 8° and 8(G,) 2t for lsSi< sr, 


Lemmas A.3, A.5, and A.6 can be cambined into 


A.7 Lemma 


Let F be an (6,n,k)-formula. Then for any t 2 1 and ae Dif 


n 2 Tle (C+) 7", (ctiatne) 


(see Lemma A.6 for the value of c), there exists a subset X © S(F) 
such that either 
(1) FS is equivalent to a PC of the formulas Fyocee oF, over 3° 
where r S$ Nax? each variable of Fy occurs at most k-l times in it, 
and Fy contains at least t variables of X or 
(2) Fe is equivalent to an SC of an e, component K over 3° 
with a formula G over 3° (through the input variable) such that S(G) 
is the set of the lateral variables of K and no variable occurs more 
than k~l times in G; or to an e, “component K over 8° with a constant 


input operator. 


A.8 Lemma 


Let F be a (¢,n,k)-formula. Then for any t 2 1 and a € D if 


n2 N5(t,k) 
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then there exists a subset X © S(F) such that Fe is equivalent to an 
SPCeC over 8° G such that (1) G has s a components, (2) each 
component is of length 2 t, and (3) the terminal components of G 


have constant input operators. 


Proof 
T(t 51) 7 ne ox In this case T(F) has at least one branch 
connected to t+l variable symbols (k=l and thus all variable symbols 
are distinct) at different nodes. This branch can be converted into 
an e, “component with constant input operator. The idea is illustrated 


in Fig. A.1. 


ny Cesk) = Me (Cet) 2 OADM CEH) | fete Tce sbdee 


We can apply Lemma A.7. The result is either (1) an e-component K 

of the correct length and constant input operator, (2) an SC of an 
e-component over 8° of the correct length and a formula to which we 

can apply the inductive hypothesis, and (3) a PC of formulas to which 
we can apply the inductive hypothesis. In each case we obtain an SPCeC 


with the desired properties. 


A.9 Lemma 


Let F be a (6,nk)-formula, Then for any t 21 and aeDif 


n 2 Tp (tsk) 
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there exists a subset X © S(F) such that F is equivalent to an SPCeC 
over 87 G such that (1) G has $ k components, (2) each component has X 
as the set of its lateral variables, (3) the terminal components have 


constant input operators and (4) Ix| = ts 


Proof 
nk nX aX 
Set Tg (tk) = TN (s-t,k) where s ={ max +{ maxf +...4+ | max 
k k-1 1 
Apply Lemma A.8 to obtain a SPCeC G' with all components having length 
2st. Since each variable appears at most k times, it can occur in 
at most k components. s is the number of nonempty subsets of <= k elements. 
Thus, if the number of variables is as indicated we are sure to kind 
in G' a subset of t variables that all occur in the same set of components 


of G'. After performing an a-merger with this set as basis, we obtain 


the desired SPCeC G. a 
Remarks on the bounds in Lemmas A.3-A.9. If 1, is approximated by 
na then 1, is inductively defined as follows: 
t 
Hz tt.4) ~ Tnax 
, ee 
k+1 
No (tk) Fy yay “Nl (t,k 1) b 
b* k times 


for a certain constant y. Thus we see that Tig (tk) 2 texp(b,2k) = b 
for k 2 k(b) for any constant b (t has not been included in the estimate 


because in applications it is constant). 
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Conversion of a formula F where each variable occurs only once 
into an equivalent e-component by setting certain variables to a. 


Fig. A.l 
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APPENDIX B 


THE LENGTH OF THE MOD 2 SUM OVER nt 


There is an isomorphism between the set of formulas over II and series- 
parellel contact networks. We assume the reader is familiar with this model 
as well as with the isomorphism in question. In this case if F is a formula 
over II, then L(F) corresponds to the number of contacts in the network corres- 
ponding to F. 

For convenience, we will derive the result in contact network terminology. 

Given a (series-parallel) contact network C, a chain is set of contacts 
such that when they are all closed, C conducts (we will say "C is 1"); a cut 


set is a set of contacts such that when they are all open, C does not conduct 


(we will say "C is 0"). In the obvious way, we define minimal chain, minimal 


cut set (i.e., when one contact is deleted the corresponding property does not 


hold). 


B.1 Lemma 
Given a contact network C and any minimal chain and minimal cut set, their 


intersection is a singleton. 


This result is due to Khrapchenko [Kh71]. 
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Proof 

By induction on the number m of contactsin C. For m = 1 the assertion 
is obviously true. If ms 1, C must be either a series combination of smaller 
networks Cy and Cy» or a parallel combination of smaller networks C, and C,. 


1 2 


In each case it is simple to establish the lemma. O 


n 
Suppose now we have a contact network S that represents © Xo Let a 


i=l 
denote the number of contacts labeled with x. or x, . Then we are interested 
A J J 
in dX m,. 
jel 


Consider n-tuples (a,) for 1 Si <n and a, € {0,1}. An n-tuple of this 
kind will be called even if it has an even number of 1's,otherwise it is odd. 
Obviously S must be 1 on odd n«tuples and O on even ones. 


Consider an arbitrary odd n-tuple a = (ayseresa peoera) and an even 


i 
n=tuple b = (by sees sds aeee sb.) at Hamming distance 1 from a. If b, = ays then 


i 

all other components of a and b are equal. gy will denote the n-tuple with 
a single 1 in the a place. Then we will write b =a ® er. 

To each odd n-tuple a we can assign a minimal chainc(a) (consisting of 
a subset of contacts of S that are closed at a and that do form a minimal 
chain); similarly, to each even n-tuple b we can assign a minimal cut set 
s(b) (consisting of a set of contacts of S that are open at b and that do form 
a minimal cut set). 

Let a be odd, b=a Ce even. Then by Lemma B.1, c(a) [ s(b) is a 
singleton; in fact, it is easy to verify that it must be a contact labeled 


either with Xs or Kye 
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We build now Tables I and II. The rows of Table I correspond to odd 
n=tuples while those of Table II correspond to even n-tuples. Thus both 
have gred rows. The columns of both tables correspond to the variable Xp 
for 1 Si<n. The entry 4(a,j) in Table I is c(a) [ s(a *e,) This entry 
will be represented by a number between 1 and mss 

Let th denote the number of times contact number i (among those labeled 
with x, or x5) appears in column j of Table I. Then 
a 

z 

i=l 


ti = 2 (B. 1) 
The entry B(b,j) of Table II is s(b) Nc(b ®e,). (B.1) again holds. 
Construct now Table III. The rows of Table III correspond to all possible 
pairs (a,b) where a and b are odd and even n-tuples respectively. The columns 
of Table III again correspond to variables. An entry of Table III is y(a,b,j) = 
(a(a,j), B,i)- 
Consider now the diagonal entries of Table III (i.e., (4,8) such that 
a= 68). Let (A(a,j), B(b,j)) be such an entry. Then G(a,j) = B(b,j) = 


c(a) % s(b). Thus, by Lemma B.1, there can be only one such entry in a row. 


n mM 
2 
The number of diagonal entries is 2 oS :t,.4 hus, 

: : ij 

jal i=1 
n 7 
he ee eee (B.2) 
j=l i=l J 


Combining a version of Cauchy's inequality 


Mm. m. 
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